Skip to main content
Download PDF
Download ePub

Fidelity to the ACT SMART Toolkit: an instrumental case study of implementation strategy fidelity

Implementation Science Communications volume 4, Article number: 52 (2023) Cite this article

Abstract

Background

Evidence-based practices (EBPs) are shown to improve a variety of outcomes for autistic children. However, EBPs often are mis-implemented or not implemented in community-based settings where many autistic children receive usual care services. A blended implementation process and capacity-building implementation strategy, developed to facilitate the adoption and implementation of EBPs for autism spectrum disorder (ASD) in community-based settings, is the Autism Community Toolkit: Systems to Measure and Adopt Research-based Treatments (ACT SMART Toolkit). Based on an adapted Exploration, Adoption decision, Preparation, Implementation, Sustainment (EPIS) Framework, the multi-phased ACT SMART Toolkit is comprised of (a) implementation facilitation, (b) agency-based implementation teams, and (c) a web-based interface. In this instrumental case study, we developed and utilized a method to evaluate fidelity to the ACT SMART Toolkit. This study responds to the need for implementation strategy fidelity evaluation methods and may provide evidence supporting the use of the ACT SMART Toolkit.

Methods

We used an instrumental case study approach to assess fidelity to the ACT SMART Toolkit during its pilot study with six ASD community agencies located in southern California. We assessed adherence, dose, and implementation team responsiveness for each phase and activity of the toolkit at both an aggregate and individual agency level.

Results

Overall, we found that adherence, dose, and implementation team responsiveness to the ACT SMART Toolkit were high, with some variability by EPIS phase and specific activity as well as by ASD community agency. At the aggregate level, adherence and dose were rated notably lowest during the preparation phase of the toolkit, which is a more activity-intensive phase of the toolkit.

Conclusions

This evaluation of fidelity to the ACT SMART Toolkit, utilizing an instrumental case study design, demonstrated the potential for the strategy to be used with fidelity in ASD community-based agencies. Findings related to the variability of implementation strategy fidelity in the present study may also inform future adaptations to the toolkit and point to broader trends of how implementation strategy fidelity may vary by content and context.

Peer Review reports

Background

Autism spectrum disorder

An autism spectrum disorder (ASD) affects approximately 1 in 44 children in the USA and has been identified as a public health concern estimated to cost 461 billion dollars a year for services and treatment by 2030 [1,2,3]. ASD is characterized by social and communication difficulties as well as restricted and repetitive behaviors and interests. Further, ASD commonly co-occurs with anxiety disorders, obsessive–compulsive disorder, attention deficit hyperactivity disorder, and/or oppositional defiant disorder [4,5,6]. Additionally, children on the autism spectrum have higher rates of behaviors such as self-injury, aggression, tantrums, and property destruction compared to neurotypical peers [7,8,9].

Both the core features and co-occurring disorders and behaviors of ASD have been found to predict unsatisfactory outcomes in quality-of-life factors. This includes peer relationships, educational attainment, employment, and independent living in adulthood [5, 10, 11]. Associations between autisticFootnote 1 characteristics and unsatisfactory quality-of-life outcomes are also maintained by systemic barriers to the inclusion of autistic individuals. These barriers include societal stigma and lack of appropriate accommodations in education, employment, and housing opportunities [12,13,14].

The prevalence rate for ASD continues to grow dramatically as practices for diagnosis improve [3, 15]. However, despite their potential to improve outcomes for autistic youth and reduce individual and societal costs [16,17,18], barriers to community-level identification and intervention remain [3, 19]. Evidence-based practices (EBPs) have been shown to improve a variety of outcomes for autistic children. However, EBPs are often inconsistently implemented or mis-implemented in community-based settings where many autistic children receive services [20,21,22,23,24]. As a result, there is a considerable number of children on the autism spectrum not receiving therapeutic practices empirically demonstrated to improve outcomes as part of their usual care. Thus, there is a need to identify, develop, and evaluate strategies facilitating the adoption, implementation, and sustained use of EBPs for ASD within community settings.

ACT SMART implementation toolkit

Drahota and colleagues [20, 25, 26] developed a packaged implementation process tool designed to facilitate autism EBP adoption, preparation, uptake, and sustained use in autism community-based agencies. The Autism Community Toolkit: Systems to Measure and Adopt Research-based Treatments (ACT SMART Toolkit) was developed through a review of existing implementation strategy taxonomies and evidence [27, 28] and by incorporating insights from collaborative community and academic partners through a community-academic partnership [29]. The ACT SMART Toolkit was developed to have steps and activities aligned with the multi-phased Exploration, Preparation, Implementation, Sustainment (EPIS) implementation framework that was adapted for this setting [20, 30, 31].

The explicit goal of the ACT SMART Toolkit during development was to co-create a systematized, yet flexible, process and accompanying set of tools that would facilitate the adoption, implementation, and sustainability of ASD EBPs within community settings [27, 30]. Specifically, the ACT SMART Toolkit is comprised of three implementation strategies: implementation facilitation, agency-based implementation teams (e.g., capacity-building implementation strategy), and a web-based interface (e.g., implementation process strategy) that provides access to the steps and activities that facilitate momentum within and between implementation phases [25,26,27,28, 30].

In practice, the ACT SMART Toolkit and implementation facilitator guide ASD agency implementation teams to explore their agency’s receptivity to implementing a new EBP, identify and decide upon an EBP that meets their agency’s needs, prospectively plan to implement the EBP (e.g., adaptation, training, discrete implementation strategy use), evaluate the EBP implementation process, and develop a plan for EBP sustainment (see Fig. 1; [25]). Of note, the ACT SMART Toolkit was designed to build capacity within agencies to utilize a systematic implementation process and was developed to be used flexibly (e.g., move backward, skip activities or steps, etc.) to meet the specific needs of individual ASD agencies. That is, the ACT SMART Toolkit was designed to allow for individualization and flexibility within a structured set of phases, steps, and activities [26].

Fig. 1
figure 1

ACT SMART Implementation Toolkit steps and activities to support EBP implementation

Full size image

Importantly, the ACT SMART Toolkit has been pilot tested with six ASD community-based agencies. Preliminary work by Drahota and colleagues suggests that the toolkit is feasible, acceptable, and useful to agency implementation teams [Drahota A, Meza R, Martinez JI, Sridhar A, Bustos TE, Tschida J, Stahmer A, Aarons GA: Feasibility, acceptability, and utility of the ACT SMART implementation toolkit, in preparation]. In addition, Sridhar and Drahota [32] found that the toolkit facilitates clinically meaningful changes in agency provider- and supervisor-reported EBP use. Moreover, Sridhar and colleagues [33] identified salient facilitators (i.e., facilitation, facilitation meetings, and phase-specific activities) and salient barriers (i.e., website issues, perceived lack of resources, and contextual factors within ASD community agencies such as time constraints and funding) to the utilization of the ACT SMART Toolkit in the pilot study. Together, these findings suggest that the ACT SMART Toolkit may facilitate the adoption and implementation of ASD EBPs within community-based settings but likely needs revision to overcome factors that may limit its effectiveness. Appropriately powered quasi- or experimental research, necessary to test the toolkit’s effectiveness, will require the assessment and reporting of implementation strategy fidelity per the standards for reporting implementation studies [34].

Implementation strategy fidelity

Fidelity is a construct that assesses the extent to which individuals (e.g., providers) deliver a strategy as planned [35,36,37]. Researchers have proposed components that contribute to fidelity should include (1) adherence to the outlined procedures, (2) proportion of the strategy received (i.e., dose), (3) extent of individual responsivity to the strategy (i.e., participant responsiveness), (4) quality of implementation, and (5) differentiation from unspecified procedures [38, 39]. Researchers have also proposed that quality and differentiation primarily capture the characteristics of an EBP being implemented whereas adherence, dose, and participant responsiveness are particularly relevant fidelity constructs for implementation strategy fidelity [37, 40].

Dusenbury [38] defines adherence as the extent to which activities are consistent with the way a strategy is proposed, dose as the amount of strategy content received by participants, and participant responsiveness as the extent to which participants are engaged by and involved in the strategy. In relation to the fidelity to implementation strategies, including implementation process strategies and capacity-building implementation strategies, participants could refer to agency implementation teams. Agency implementation teams are groups of individuals within an agency responsible for guiding EBP implementation [41].

Fidelity is also considered dynamic and may be influenced by factors such as provider characteristics, the setting, and/or complexity of the strategy [37, 42]. Assessing implementation strategy fidelity, especially to implementation process and blended implementation strategies, may help implementation strategy developers further understand which components of an implementation strategy may be core functions needed to produce desired outcomes and which may be adapted to account for varying contextual characteristics [43,44,45]. Assessing fidelity may also improve the generalizability and reproducibility of implementation strategies [46]. Of course, this is contingent upon an ability to determine whether implementation of the strategy remained consistent with its underlying theory [47, 48]. Notably, increasing understanding about how implementation strategies work has been identified as a research priority within the field of dissemination and implementation science [49,50,51].

Despite its importance, fidelity to implementation strategies, including implementation process and capacity-building implementation strategies [27, 46], has rarely been assessed; instead, research has often focused only on fidelity to the EBPs being implemented [37, 52]. Indeed, Slaughter et al. [37] conducted a scoping review that indicated no articles reporting fidelity to implementation strategies included definitions or conceptual frameworks for assessing implementation strategy fidelity. To our knowledge, few studies have examined fidelity to an implementation strategy and only one recent study has used a guiding theoretical framework [52, 53].

Present study

This study utilized an instrumental case study approach to assess fidelity to the ACT SMART toolkit during its pilot study to extract insights into the use of the toolkit as well as implementation strategy fidelity methods more broadly [54]. Examining implementation strategy fidelity can provide insight into the overall potential for ASD community-based agencies to use the toolkit as planned and, if effective, ultimately implement and sustain EBP use. This information may be particularly useful for implementation practitioners using the toolkit with ASD community-based agencies in the future and needing to discern when fidelity or adaptation to toolkit activities is most appropriate. Indeed, ASD community-based agencies may have competing priorities and contextual barriers to completing the toolkit in its entirety with fidelity [33]. Further, this study provides one of the first process models to assess fidelity to a packaged implementation process tool comprised of multiple implementation strategies. This model may inform a broader understanding of implementation strategy fidelity and contribute to underlying theory. Specifically, we examined fidelity to the ACT SMART Toolkit at an aggregate and individual agency level according to adherence, dose, and participant responsiveness during its pilot study.

Methods

Participants

A total of six ASD community agencies located in Southern California were included in the pilot study of the ACT SMART toolkit. Four of the ASD community agencies were Applied Behavior Analysis (ABA) organizations, one was an ABA and mental health organization, and one agency was a Speech and Language Pathology organization. Participating agency leaders (n = 6) attended an ACT SMART Toolkit orientation meeting describing implementation science concepts and the ACT SMART Toolkit components; two of the agency leaders had been involved in the community-academic partnership that advised on the development of the Toolkit. Prior to attending an orientation for the ACT SMART Toolkit, agency leaders completed self-report measures of implementation and ACT SMART Toolkit knowledge. Rated on a 5-point Likert scale (0 = “Not at all knowledgeable” to 4 = “Extremely knowledgeable”), descriptive analyses indicate that they had moderate knowledge of implementation, generally (Median = 3.0), expected implementation outcomes (Median = 2.0), implementation barriers (Median = 3.0), the purpose of implementation (Median = 2.0), and the purpose of the ACT SMART Toolkit (Median = 2.0), activities (Median = 1.0), implementation teams (Median = 1.0), and facilitation meetings (Median = 1.0) [30].

Thereafter, agency leaders developed agency-based implementation teams composed of agency staff (Table 1 provides implementation team demographic and discipline information). At least one agency leader was required for each implementation team to ensure that adoption and implementation planning decisions could be made without additional approvals. Eligibility criteria included (1) holding the role of CEO, director, or leading decision-maker regarding treatment use at an ASD community agency eligible to participate in the ACT SMART pilot study; (2) willingness to participate in the pilot study for 1 year; and (3) agreement to provide feedback after completing each phase of the pilot study. The agency leader for each participating agency then invited up to four other agency staff members (i.e., supervisors and direct providers) to complete their agency’s implementation team. Eligibility criteria for implementation team members were to agree to complete the toolkit and provide feedback about its feasibility, acceptability, and utility.

Table 1 Demographic and discipline information across implementation teams
Full size table

Five of the six ASD community agencies completed all phases of the ACT SMART toolkit. These agencies each chose to adopt the EBP, Video Modeling, from a menu of three EBPs (for study details, see [30]). One ABA agency chose not to adopt an EBP at the end of the adoption decision phase of the toolkit because the implementation team did not find any of the EBPs to meet the needs of the agency (e.g., lack of agency-EBP fit).

Materials and procedure

As part of the pilot study, a research assistant served as an independent observer and evaluated implementation teams’ fidelity using the Implementation Milestones form, adapted with permission from the Stages of Implementation Completion [55], and the ACT SMART Activity Fidelity form (Drahota A, Martinez JI: ACT SMART Milestones and Activity Fidelity Forms, unpublished). The ACT SMART Implementation Milestones form required the independent observer to record a Yes or No answer (scored as 1 and 0, respectively) for whether activities during pre-implementation and phase 1 through phase 4 of the ACT SMART Toolkit were completed (see Additional file 1: Appendix A). Scores were converted into percentages to assist with interpretation. The ACT SMART Activity Fidelity form presented more detailed questions regarding completion of activities during Phase 2: Adoption; Phase 3: Preparation; and Phase 4: Implementation. The independent observer recorded a Yes or No answer (scored as 1 and 0, respectively) for whether implementation teams completed each activity and then rated the amount of the form completed using a 4-point Likert scale where 0 = “Nothing Completed”, 1 = “Minimally Completed (1–2 items)”, 2 = “Moderately Completed (3–4 items)”, and 3 = “Mostly/All Completed (5–6 items)” (see Additional file 1: Appendix B).

In addition to the observational data collected using the ACT SMART Implementation Milestones form and the ACT SMART Activity Fidelity form, ACT SMART facilitators rated implementation team engagement using the ACT SMART Implementation Team Engagement Rating Scale that was created by the toolkit developers. Immediately after each facilitation meeting, the ACT SMART facilitator(s) rated implementation team engagement in ACT SMART activities and facilitation meetings since the last facilitation meeting occurred. Engagement ratings were completed using a 5-point Likert scale where 1 = “Not at all engaged”, 2 = “Slightly Engaged”, 3 = “Moderately Engaged”, 4 = “Very Engaged”, and 5 = “Extremely Engaged” (see Additional file 1: Appendix C).

In the present study, we used the operational definitions from Dusenbury [38] and an overall scoring rubric for implementation strategy fidelity developed by Slaughter et al. [37] as the basis for using the ACT SMART Implementation Milestones form, ACT SMART Activity Fidelity form, and ACT SMART Implementation Team Engagement Rating Scale to assess implementation strategy fidelity via adherence, dose, and participant responsiveness, respectively.

Analysis plan

We used an instrumental case study approach to explore both fidelity to the ACT SMART Toolkit and potential generalizations to a broader underlying theory of implementation strategy fidelity. The Standards for Reporting Implementation Studies (StaRI) checklist was used to assist reporting, given that the ACT SMART Toolkit is a packaged, blended implementation process tool developed to increase EBP use in ASD community agencies [Drahota A, Meza R, Martinez JI, Sridhar A, Bustos TE, Tschida J, Stahmer A, Aarons GA: Feasibility, acceptability, and utility of the ACT SMART implementation toolkit, in preparation]. First, we assessed adherence, dose, and participant responsiveness for the ACT SMART Toolkit overall as well as for each phase and activity of the toolkit. Utilizing the ACT SMART Implementation Milestones form, we assessed adherence via a Yes/No answer to whether implementation milestones were completed. Overall, by phase, and by activity, we calculated the average percentage of “Yes” answers for required toolkit activities. We assessed dose by analyzing Likert scales on the ACT SMART Activity Fidelity form evaluating how much of each activity was completed. Overall, by phase, and by activity, we calculated the median dose rating. Finally, we assessed participant responsiveness by analyzing the Likert scales on the ACT SMART Implementation Team Engagement Rating Scale and used dates of completion to confirm phase. Overall and by phase, we calculated the median participant responsiveness rating. We did not calculate the median participant responsiveness rating by activity as ratings for engagement were only given by phase. We also calculated an average percent agreement on participant responsiveness ratings from facilitation meetings in which multiple facilitators were present. All facilitators attended informal trainings on rating participant responsiveness using the ACT SMART Implementation Team Engagement Scale. During supervision sessions with facilitators, facilitators engaged in discussions about their rationale for participant responsiveness ratings for each facilitation meeting. Lastly, we calculated overall, phase, and activity adherence, dose, and participant responsiveness for each agency implementation team.

Results

Aggregate fidelity to the ACT SMART Toolkit

Agency implementation teams adhered to an overall average of 90% (SD = 11.3%) of ACT SMART Toolkit activities. Average adherence ranged from 74% (SD = 19.5%) completion of toolkit activities during the preparation phase of the toolkit to 100% (SD = 0%) completion of toolkit activities during the implementation phase of the toolkit (see Table 2). While the completion rate for individual activities within phases was also relatively high across agencies, there was some variability.

Table 2 Adherence to the ACT SMART Implementation Toolkit in aggregate and by individual agency implementation team
Full size table

Related to dose, the independent observer gave agency implementation teams an overall median rating of “Mostly/All Completed” (Median = 3.0). The lowest median dose rating was between “Moderately Completed” to “Mostly/All Completed” (Median = 2.5) during the preparation phase whereas the highest median dose ratings were “Mostly/All Completed” (Median = 3.0) during the adoption and implementation phases of the toolkit (see Table 3). Consistent with observations of adherence, there were lower dose ratings for activities such as the benefit–cost estimator, gathering treatment materials, and developing adaptation and implementation plans compared to higher completion rates for activities related to treatment evaluation, funding, training, and carrying out developed plans. Here, it should be noted that dose ratings by activity could not be calculated for the implementation phase given that evaluation surveys during this phase were designed to be dynamic and capture completion of individualized sets of tasks by agency [Drahota A, Meza R, Martinez JI, Sridhar A, Bustos TE, Tschida J, Stahmer A, Aarons GA: Feasibility, acceptability, and utility of the ACT SMART implementation toolkit, in preparation].

Table 3 Dose to the ACT SMART Implementation Toolkit in aggregate and by individual agency implementation team
Full size table

For participant responsiveness, ACT SMART facilitators rated agency implementation teams with a median rating falling between “Moderately Engaged” and “Very Engaged” (Median = 3.8). The lowest median participant responsiveness rating was between “Moderately Engaged” and “Very Engaged” (Median = 3.5) during the adoption decision phase of the toolkit. The highest median participant responsiveness rating was at “Extremely Engaged” (Median = 5.0) during the implementation phase (see Table 4). For facilitation meetings with multiple ACT SMART facilitators present, there was a 92.43% average agreement on participant responsiveness ratings.

Table 4 Participant responsiveness to the ACT SMART Implementation Toolkit in aggregate and by individual agency implementation team
Full size table

Individual agency fidelity to the ACT SMART Toolkit

Across agencies, there was generally high adherence to toolkit activities; the lowest agency implementation team adhered to an overall average of 85.3% (SD = 20.2%) of toolkit activities (Table 2). While there was some variability in adherence across phases and activities by agency, there was no readily identifiable pattern of agencies consistently having lower or higher adherence compared to other agencies. Consistent with other results, the preparation phase appeared to have the lowest adherence ratings across agencies.

Agencies also all had generally high dose ratings for toolkit activities, except for the one agency (Agency 3) that chose not to adopt an EBP at the end of Phase 2: Adoption (Table 3). Like the ratings of adherence by agency, there was variability in dose ratings but no consistent identifiable patterns. Further, the preparation phase had the lowest dose ratings across agencies.

Consistent with both observations of adherence and dose ratings across agencies, all agencies also had relatively high ratings of participant responsiveness (Table 4). The agency with the lowest median participant responsiveness rating was rated between “Moderately Engaged” to “Very Engaged” (Median = 3.1). However, in contrast to observations of adherence and dose ratings, agencies did not appear to have lower participant responsiveness during the preparation phase compared to other toolkit phases.

Discussion

Fidelity to the ACT SMART Toolkit

Our investigation used an instrumental case study approach to evaluate implementation strategy fidelity to the ACT SMART Toolkit by assessing observational descriptive ratings of adherence, dose, and participant responsiveness. Our evaluation provides one of the first models of assessing fidelity to a blended implementation process and capacity-building implementation strategy. In addition, our evaluation provides important insights into both the potential for ASD community-based agencies to use the toolkit effectively and implementation strategy fidelity more broadly. Overall, we found that adherence, dose, and participant responsiveness to the ACT SMART Toolkit were relatively high, which supports the potential for the toolkit to be used with fidelity in ASD community agencies. Despite their potential to improve outcomes for a growing clinical population, EBPs for ASD are often inconsistently or mis-implemented in community settings. Thus, understanding the effective use of implementation strategies, such as the ACT SMART Toolkit, could contribute to reducing the EBP research-to-practice gap [20,21,22,23,24].

Although we found fidelity to be high overall, there was some variability in implementation strategy fidelity by toolkit phase. Specifically, we found that adherence and dose were rated the lowest in the preparation phase (Phase 3) at an aggregate level. However, we were underpowered to determine whether the differences by phase were statistically significant. One possible rationale for the descriptive finding of lower adherence and dose in Phase 3 is that there were substantial differences in the cognitive or informational demands of toolkit activities by phase. Indeed, the preparation phase required gathering materials, evaluating prospective adaptations, and developing training and adaptation plans whereas the implementation phase required carrying out and evaluating the developed plans. Notably, there were both lower adherence and dose ratings for toolkit activities such as developing adaptation and implementation plans compared to toolkit activities related to evaluating treatments, funding, and training. Thus, the lower adherence and dose in the preparation phase may reflect the need to reduce the amount or intensity of toolkit activities to better align with ASD community agencies’ capacity to plan for implementation. Considering recently identified context-specific barriers and facilitators to the ACT SMART Toolkit, such as availability of funding, time, and staffing, would also likely be critical to enhancing the toolkit overall [33, 56].

Another potential rationale for lower adherence and dose during the preparation phase may be that ASD community agencies perceived greater value in implementing the chosen EBP than in planning for its implementation. While agency implementation teams were rated as moderately to very engaged during the preparation phase, it is unclear how well facilitators were able to emphasize the important relationship between planning and implementation. However, researchers have recently proposed that fostering this understanding is necessary to support successful and sustainable implementation [57]. Thus, the ACT SMART Toolkit may also benefit from incorporating a greater focus on the practical importance of planning for implementation of EBPs.

Implementation strategy fidelity theory

Our instrumental case study assessment of fidelity to the ACT SMART Toolkit within ASD community agencies notably provides one of the first process models of assessing blended implementation strategy fidelity. Although a considerable amount of research has been conducted on intervention fidelity, few researchers have explored implementation strategy fidelity [37, 52, 53]. For example, Slaughter et al. [37] found that no studies reporting on fidelity to implementation included a specific definition or theoretical framework for assessing implementation strategy fidelity. To our knowledge, only Berry and colleagues [52] recently adapted the Conceptual Framework for Implementation Fidelity to guide their evaluation of fidelity to practice facilitation as a strategy to improve primary care practices’ adoption of evidence-based guidelines for cardiovascular disease.

Despite limited research, evaluating and understanding implementation strategy fidelity have important implications and are identified as research priorities within dissemination and implementation science [47,48,49,50,51]. High fidelity to an implementation strategy may be reflective of other important implementation outcomes, such as high acceptability, appropriateness, and feasibility [58, 59]. Further, implementation strategy fidelity may inform the determination of which components of a strategy are required to produce change (e.g., core components) and which can be removed or adapted to account for varying contextual characteristics [43,44,45]. This knowledge may allow for demand optimization when the implementation strategy is being used, which may be particularly important when users of an implementation strategy have competing priorities or contextual factors that make completing the entirety of a blended implementation strategy difficult [33].

From our instrumental case study of ACT SMART Toolkit fidelity, we have demonstrated that fidelity to blended implementation strategies, including implementation planning strategies and capacity-building strategies, is possible. Further, implementation strategy fidelity may vary according to differing components of a strategy, such as components focusing on preparation for implementation versus components focusing on implementation itself. We also observed that implementation strategy fidelity may vary by context. Here, implementation strategy fidelity was observed to vary across different ASD community agencies using the ACT SMART Toolkit. These findings suggest that a next step to further understand implementation strategy fidelity may be investigating shifts across both strategy content and context. Importantly, increasing this understanding could then also inform commonly needed adaptations to improve implementation strategy fidelity.

Strengths

We propose that the main strength of our investigation is that we demonstrate one of the first instrumental case studies to consider fidelity to a blended implementation strategy. Importantly, our assessment of fidelity to the ACT SMART Toolkit may be able to provide a framework for other evaluations of implementation strategy fidelity and inform the underlying theory of implementation strategy fidelity. Within our evaluation, we also importantly found overall high fidelity to the toolkit within ASD community-based agencies and identified potential ways in which to optimize demands of the toolkit and increase sustainability. Understanding fidelity to the toolkit within a pilot study is a critical first step before broader use with many agencies in appropriately powered studies.

Limitations

In contrast, important limitations of our investigation include potential issues with measurement of specific implementation strategy fidelity variables. For example, Berry and colleagues [52] recently considered participant responsiveness as a moderator of implementation strategy fidelity rather than a component of fidelity itself as considered in our analysis. Moreover, the potential issues with measurement may have been compounded by the fact that standard measures were not used for dose and participant responsiveness. However, as an emerging field, implementation science often faces issues related to measurement and standardized measures specific to implementation strategy fidelity have not yet been developed [49, 50, 60]. Researchers have developed some standard measures for intervention fidelity, and these may be able to be adapted to assess implementation strategy fidelity in the future [61].

Another potential limitation in our investigation is that there were different raters for adherence, dose, and participant responsiveness. While an independent observer rated adherence and dose for each implementation team, participant responsiveness was rated by a facilitator following implementation teams’ facilitation meetings. Although this presents potential for bias, direct observation by independent observers and even implementers has been found to be more accurate than collecting reports directly from participants [61]. Further, when two facilitators independently gave ratings for participant responsiveness, there were high rates of agreement. Ratings were also only given for implementation teams as one unit rather than individually for each implementation team member. In the present study, rating at the level of implementation teams was practical given that ASD community agencies may face high rates of staff turnover [33]. However, future research would benefit from examining whether implementation strategy fidelity varies by implementation team member or staff role.

Moreover, while we were generally able to assess implementation strategy fidelity by toolkit phase and activities, we were unable to assess all variables for all activities and by toolkit facet (i.e., web-based interface versus facilitation meetings). Thus, we are unable to make conclusions about all activities and the impact of the blended nature of the toolkit on implementation strategy fidelity. Further, our results may not generalize to discrete implementation strategies, which may benefit from their own instrumental case studies.

Lastly, the most important limitation of our assessment of fidelity to the ACT SMART Toolkit was the limited sample size that rendered us under-powered to fully evaluate relationships between implementation strategy fidelity and EBP use. Moreover, in the limited sample, implementation teams that completed each phase of the toolkit all chose to adopt video modeling. While this may reflect the particular ease of adopting video modeling (e.g., low training requirements and cost), it is unclear whether results would vary with a different choice of EBP. Our limited sample size also precluded us from considering additional factors such as implementation team and provider demographics and organizational climate within ASD community agencies. While we were able to observe variable implementation strategy fidelity across ASD community agencies, we were not yet able to identify consistent patterns related to higher or lower implementation strategy fidelity. However, there is evidence that some of these factors may moderate the relationship between implementation strategy fidelity to the ACT SMART toolkit and increased EBP use [62].

Future research would benefit from consideration of potential moderators of implementation strategy fidelity and utilizing standard measures and independent raters [60,61,62,63,64,65]. In addition, future studies may benefit from a design intended to systematically evaluate fidelity to all components of a strategy. These lines of research may provide further insight into both effective use of the ACT SMART Toolkit and advancing the field of implementation science more broadly.

Conclusions

By utilizing an instrumental case study approach, we advanced understanding of effective use of the ACT SMART Toolkit as well as the theory of implementation strategy fidelity more broadly. We found that the ACT SMART Toolkit has the potential to be used with high fidelity in ASD community-based agencies. However, we also found that there was some variability in fidelity among toolkit phases, which points to possible adaptations needed to improve toolkit use even further. Considering adaptations may be critical as these findings may reflect that fidelity to blended implementation strategies is dynamic and affected by both strategy content and context. By increasing the use of and fidelity to effective implementation strategies that facilitate EBP adoption, utilization, and sustainment within community-based settings, there is potential to increase overall public health.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the senior author (AD) upon reasonable request.

Notes

  1. We use “identity-first” language in some instances due to recent studies indicating that identity-first language is preferred by some autistic individuals and a recent review highlighting potentially ableist language in autism research [12].

Abbreviations

ASD:

Autism spectrum disorder

EBP:

Evidence-based practice

ACT SMART Toolkit:

Autism Community Toolkit: Systems to Measure and Adopt Research-Based Treatments

EPIS model:

Exploration, Preparation, Implementation, Sustainment model

ABA:

Applied Behavior Analysis

References

  1. Blaxill M, Rogers T, Nevison C. Autism tsunami: the impact of rising prevalence on the societal cost of autism in the United States. J Autism Dev Disord. 2022;52(6):2627–43. https://doi.org/10.1007/s10803-021-05120-7.

    Article  PubMed  Google Scholar 

  2. Leigh JP, Du J. Brief report: Forecasting the economic burden of autism in 2015 and 2025 in the United States. J Autism Dev Disord. 2015;45(12):4135–9. https://doi.org/10.1007/s10803-015-2521-7.

    Article  PubMed  Google Scholar 

  3. Maenner MJ, Shaw KA, Bakian AV, Bilder DA, Durkin MS, Esler A, et al. Prevalence and characteristics of autism spectrum disorder among children aged 8 years — autism and developmental disabilities monitoring network, 11 Sites, United States, 2018. MMWR Surveill Summ. 2021;70(11):1–16. https://doi.org/10.15585/mmwr.ss7011a1.

    Article  PubMed  PubMed Central  Google Scholar 

  4. American Psychiatric Association. Autism spectrum disorder. In: Diagnostic and statistical manual of mental disorders. 5th ed. Arlington: American Psychiatric Association; 2013. https://doi.org/10.1176/appi.books.9780890425596.

    Chapter  Google Scholar 

  5. Lai MC, Lombardo MV, Baron-Cohen S. Autism. Lancet. 2014;383(9920):896–910. https://doi.org/10.1016/S0140-6736(13)61539-1.

    Article  PubMed  Google Scholar 

  6. Simonoff E, Pickles A, Charman T, Chandler S, Loucas T, Baird G. Psychiatric disorders in children with autism spectrum disorders: prevalence, comorbidity, and associated factors in a population-derived sample. J Am Acad Child Adolesc Psychiatry. 2008;47(8):921–9. https://doi.org/10.1097/CHI.0b013e318179964f.

    Article  PubMed  Google Scholar 

  7. Hattier MA, Matson JL, Belva BC, Horovitz M. The occurrence of challenging behaviours in children with autism spectrum disorders and atypical development. Dev Neurorehabil. 2011;14(4):221–9. https://doi.org/10.3109/17518423.2011.573836.

    Article  PubMed  Google Scholar 

  8. Horner RH, Carr EG, Strain PS, Todd AW, Reed HK. Problem behavior interventions for young children with autism: a research synthesis. J Autism Dev Disord. 2002;32(5):423–46. https://doi.org/10.1023/A:1020593922901.

    Article  PubMed  Google Scholar 

  9. Stevens E, Atchison A, Stevens L, Hong E, Granpeesheh D, Dixon D, Linstead E. A cluster analysis of challenging behaviors in autism spectrum disorder. In: 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA). IEEE; 2017. p. 661–6.

    Chapter  Google Scholar 

  10. Kim SY, Bottema-Beutel K. A meta regression analysis of quality of life correlates in adults with ASD. Res Autism Spectrum Disorders. 2019;63:23–33. https://doi.org/10.1016/j.rasd.2018.11.004.

    Article  Google Scholar 

  11. Mason D, Mackintosh J, McConachie H, Rodgers J, Finch T, Parr JR. Quality of life for older autistic people: the impact of mental health difficulties. Res Autism Spectrum Disorders. 2019;63:13–22. https://doi.org/10.1016/j.rasd.2019.02.007.

    Article  Google Scholar 

  12. Bottema-Beutel K, Kapp SK, Lester JN, Sasson NJ, Hand BN. Avoiding ableist language: suggestions for autism researchers. Autism Adulthood. 2021;3(1):18–29. https://doi.org/10.1089/aut.2020.0014.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Pitney J. Lifetime social cost. Autism Politics and Policy. 2020. Available from http://www.autismpolicyblog.com/2020/02/lifetime-social-cost.html

  14. Robertson SM. Neurodiversity, quality of life, and autistic adults: shifting research and professional focuses onto real-life challenges. DSQ. 2010;30(1). doi: https://doi.org/10.18061/dsq.v30i1.1069

  15. King M, Bearman P. Diagnostic change and the increased prevalence of autism. Int J Epidemiol. 2009;38(5):1224–34. https://doi.org/10.1093/ije/dyp261.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Eapen V, Črnčec R, Walter A. Clinical outcomes of an early intervention program for preschool children with autism spectrum disorder in a community group setting. BMC Pediatr. 2013;13(1):3. https://doi.org/10.1186/1471-2431-13-310.1186/1471-2431-13-3.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Horlin C, Falkmer M, Parsons R, Albrecht MA, Falkmer T. The cost of autism spectrum disorders. PLoS ONE. 2014;9(9):e106552. https://doi.org/10.1371/journal.pone.0106552.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Vinen Z, Clark M, Paynter J, Dissanayake C. School age outcomes of children with autism spectrum disorder who received community-based early interventions. J Autism Dev Disord. 2018;48(5):1673–83. https://doi.org/10.1007/s10803-017-3414-8.

    Article  PubMed  Google Scholar 

  19. Elder JH, Brasher S, Alexander B. Identifying the barriers to early diagnosis and treatment in underserved individuals with autism spectrum disorders (ASD) and their families: a qualitative study. Issues Ment Health Nurs. 2016;37(6):412–20. https://doi.org/10.3109/01612840.2016.1153174.

    Article  PubMed  Google Scholar 

  20. Drahota A, Meza RD, Bustos TE, Sridhar A, Martinez JI, Brikho B, et al. Implementation-as-usual in community-based organizations providing specialized services to individuals with autism spectrum disorder: a mixed methods study. Adm Policy Ment Health. 2021;48(3):482–98. https://doi.org/10.1007/s10488-020-01084-5.

    Article  PubMed  Google Scholar 

  21. Paynter JM, Ferguson S, Fordyce K, Joosten A, Paku S, Stephens M, et al. Utilisation of evidence-based practices by ASD early intervention service providers. Autism. 2017;21(2):167–80. https://doi.org/10.1177/1362361316633032.

    Article  PubMed  Google Scholar 

  22. Pickard K, Meza R, Drahota A, Brikho B. They’re doing what? A brief paper on service use and attitudes in ASD community-based agencies. J Mental Health Res Intellect Disabil. 2018;11(2):111–23. https://doi.org/10.1080/19315864.2017.1408725.

    Article  Google Scholar 

  23. Wong C, Odom SL, Hume KA, Cox AW, Fettig A, Kucharczyk S, et al. Evidence-based practices for children, youth, and young adults with autism spectrum disorder: a comprehensive review. J Autism Dev Disord. 2015;45(7):1951–66. https://doi.org/10.1007/s10803-014-2351-z.

    Article  PubMed  Google Scholar 

  24. Wood JJ, McLeod BD, Klebanoff S, Brookman-Frazee L. Toward the implementation of evidence-based interventions for youth with autism spectrum disorders in schools and community agencies. Behav Ther. 2015;46(1):83–95. https://doi.org/10.1016/j.beth.2014.07.003.

    Article  PubMed  Google Scholar 

  25. Drahota A, Chlebowski C, Stadnick N, Baker-Ericzén MJ, Brookman-Frazee L. The dissemination and implementation of behavioral treatments for anxiety in ASD. In: Kerns C, Renno P, Storch A, Kendall PC, Wood JJ, editors. Anxiety in children and adolescents with autism spectrum disorder: evidence-based assessment and treatment. Atlanta: Elsevier; 2017. p. 231–49.

    Chapter  Google Scholar 

  26. Drahota A, Meza R, Martinez JI. The autism-community toolkit: systems to measure and adopt research-based treatments. 2014. Available from www.actsmartoolkit.com.

    Google Scholar 

  27. Leeman J, Birken S, Powell BJ, et al. Beyond “implementation strategies”: classifying the full range of strategies used in implementation science and practice. Implementation Sci. 2017;12(125):1–9. https://doi.org/10.1186/s13012-017-0657-x.

    Article  Google Scholar 

  28. Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Sci. 2015;10(21):1–14. https://doi.org/10.1186/s13012-015-0209-1.

    Article  Google Scholar 

  29. Gomez E, Drahota A, Stahmer AC. Choosing strategies that work from the start: a mixed methods study to understand effective development of community–academic partnerships. Action Research. 2021;19(2):277–300. https://doi.org/10.1177/1476750318775796.

    Article  PubMed  Google Scholar 

  30. Drahota A, Aarons GA, Stahmer AC. Developing the autism model of implementation for autism spectrum disorder community providers: study protocol. Implementation Sci. 2012;7(1):85. https://doi.org/10.1186/1748-5908-7-85.

    Article  Google Scholar 

  31. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011;38(1):4–23. https://doi.org/10.1007/s10488-010-0327-7.

    Article  PubMed  Google Scholar 

  32. Sridhar A, Drahota A. Preliminary effectiveness of the ACT SMART implementation toolkit: facilitating evidence-based practice implementation in community-based autism organizations. Int J Dev Disabil. 2022:1–6. doi:https://doi.org/10.1080/20473869.2022.2065448

  33. Sridhar A, Drahota A, Walsworth K. Facilitators and barriers to the utilization of the ACT SMART Implementation Toolkit in community-based organizations: a qualitative study. Implement Sci Commun. 2021;2(1):55. https://doi.org/10.1186/s43058-021-00158-1.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, et al. Standards for reporting implementation studies (StaRI) statement. BMJ. 2017;356: i6795.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Allen JD, Shelton RC, Emmons KM, Linnan LA. Fidelity and its relationship to implementation effectiveness, adaptation, and dissemination. In: Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and implementation research in health: Translating science to practice. Oxford Scholarship Online; 2017; 2018. https://doi.org/10.1093/oso/9780190683214.003.0016.

    Chapter  Google Scholar 

  36. Mowbray CT, Holter MC, Teague GB, Bybee D. Fidelity criteria: development, measurement, and validation. Am J Eval. 2003;24(3):315–40. https://doi.org/10.1177/109821400302400303.

    Article  Google Scholar 

  37. Slaughter SE, Hill JN, Snelgrove-Clarke E. What is the extent and quality of documentation and reporting of fidelity to implementation strategies: a scoping review. Implementation Sci. 2015;10(1):129. https://doi.org/10.1186/s13012-015-0320-3.

    Article  Google Scholar 

  38. Dusenbury L. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. 2003;18(2):237–56. https://doi.org/10.1093/her/18.2.237.

    Article  PubMed  Google Scholar 

  39. Teague GB. Fidelity. Implementation Research Institute Presentation. 2013.

    Google Scholar 

  40. Century J, Rudnick M, Freeman C. A framework for measuring fidelity of implementation: a foundation for shared language and accumulation of knowledge. Am J Eval. 2010;31(2):199–218. https://doi.org/10.1177/1098214010366173.

    Article  Google Scholar 

  41. Metz A, Bartley L. Implementation teams: a stakeholder view of leading and sustaining change. In: Implementation Science 3.0. Cham: Springer; 2020. p. 199–225.

    Chapter  Google Scholar 

  42. Cross W, West J. Examining implementer fidelity: conceptualising and measuring adherence and competence. Journal of Children’s Services. 2011;6(1):18–33. https://doi.org/10.5042/jcs.2011.0123.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Kirk MA, Haines ER, Rokoske FS, Powell BJ, Weinberger M, Hanson LC, et al. A case study of a theory-based method for identifying and reporting core functions and forms of evidence-based interventions. Translational Behavioral Medicine. 2021;11(1):21–33. https://doi.org/10.1093/tbm/ibz178.

    Article  PubMed  Google Scholar 

  44. Mihalic S. The importance of implementation fidelity. Emot Behav Disord Youth. 2004;4(4):83–105 http://www.incredibleyears.com/wp-content/uploads/fidelity-importance.pdf.

    Google Scholar 

  45. Perez Jolles M, Lengnick-Hall R, Mittman BS. Core functions and forms of complex health interventions: a patient-centered medical home illustration. J Gen Intern Med. 2019;34(6):1032–8. https://doi.org/10.1007/s11606-018-4818-7.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Chinman M, Acosta J, Ebener P, et al. “What we have here, is a failure to [replicate]”: ways to solve a replication crisis in implementation science. Prev Sci. 2022;23:739–50. https://doi.org/10.1007/s11121-021-01286-9.

    Article  PubMed  Google Scholar 

  47. The CIPHER team, Haynes A, Brennan S, Redman S, Williamson A, Gallego G, et al. Figuring out fidelity: a worked example of the methods used to identify, critique and revise the essential elements of a contextualised intervention in health policy agencies. Implementation Sci. 2015;11(1):23. https://doi.org/10.1186/s13012-016-0378-6.

    Article  Google Scholar 

  48. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258–h1258. https://doi.org/10.1136/bmj.h1258.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Akiba CF, Powell BJ, Pence BW, Nguyen MXB, Golin C, Go V. The case for prioritizing implementation strategy fidelity measurement: benefits and challenges. Transl Behav Med. 2022;12(2):335–42. https://doi.org/10.1093/tbm/ibab138.

    Article  PubMed  Google Scholar 

  50. Akiba CF, Powell BJ, Pence BW, Muessig K, Golin CE, Go V. “We start where we are”: a qualitative study of barriers and pragmatic solutions to the assessment and reporting of implementation strategy fidelity. In Review; 2022 [cited 2022 Jul 13]. Available from: https://www.researchsquare.com/article/rs-1626073/v1

  51. Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, et al. Enhancing the impact of implementation strategies in healthcare: a research agenda. Front Public Health. 2019;7:3. https://doi.org/10.3389/fpubh.2019.000031.

    Article  PubMed  PubMed Central  Google Scholar 

  52. Berry CA, Nguyen AM, Cuthel AM, Cleland CM, Siman N, Pham-Singer H, et al. Measuring implementation strategy fidelity in HealthyHearts NYC: a complex intervention using practice facilitation in primary care. Am J Med Qual. 2021;36(4):270–6. https://doi.org/10.1177/1062860620959450.

    Article  PubMed  Google Scholar 

  53. Kourouche S, Curtis K, Munroe B, Watts M, Balzer S, Buckley T. Implementation strategy fidelity evaluation for a multidisciplinary Chest Injury Protocol (ChIP). Implement Sci Commun. 2021;2(1):86. https://doi.org/10.1186/s43058-021-00189-8.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach. BMC Med Res Methodol. 2011;11(1):100. https://doi.org/10.1186/1471-2288-11-100.

    Article  PubMed  PubMed Central  Google Scholar 

  55. Chamberlain P, Brown CH, Saldana L. Observational measure of implementation progress in community based settings: The Stages of implementation completion (SIC). Implementation Sci. 2011;6(1):116. https://doi.org/10.1186/1748-5908-6-116.

    Article  Google Scholar 

  56. Powell BJ, Haley AD, Patel SV, Amaya-Jackson L, Glienke B, Blythe M, et al. Improving the implementation and sustainment of evidence-based practices in community mental health organizations: a study protocol for a matched-pair cluster randomized pilot study of the Collaborative Organizational Approach to Selecting and Tailoring Implementation Strategies (COAST-IS). Implement Sci Commun. 2020;1(1):9. https://doi.org/10.1186/s43058-020-00009-5.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Leal Filho W, Skanavis C, Kounani A, Brandli L, Shiel C, do Paço A, et al. The role of planning in implementing sustainable development in a higher education context. J Clean Prod. 2019;235:678–87. https://doi.org/10.1016/j.jclepro.2019.06.322.

    Article  Google Scholar 

  58. Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implementation Sci. 2017;12(1):108. https://doi.org/10.1186/s13012-017-0635-3.

    Article  Google Scholar 

  59. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76. https://doi.org/10.1007/s10488-010-0319-7.

    Article  PubMed  Google Scholar 

  60. Lewis CC, Dorsey C. Advancing implementation science measurement. In: Albers B, Shlonsky A, Mildon R, editors. Implementation Science 3.0. Springer Nature; 2020. p. 227.

    Chapter  Google Scholar 

  61. Ibrahim S, Sidani S. Fidelity of Intervention Implementation: A Review of Instruments. Health. 2015;07(12):1687–95. https://doi.org/10.4236/health.2015.712183.

    Article  Google Scholar 

  62. Hasson H, Blomberg S, Dunér A. Fidelity and moderating factors in complex interventions: a case study of a continuum of care program for frail elderly people in health and social care. Implementation Sci. 2012;7(1):23. https://doi.org/10.1186/1748-5908-7-23.

    Article  Google Scholar 

  63. Barber JP, Gallop R, Crits-Christoph P, Frank A, Thase ME, Weiss RD, et al. The role of therapist adherence, therapist competence, and alliance in predicting outcome of individual drug counseling: Results from the National Institute Drug Abuse Collaborative Cocaine Treatment Study. Psychother Res. 2006;16(2):229–40. https://doi.org/10.1080/10503300500288951.

    Article  Google Scholar 

  64. Hogue A, Henderson CE, Dauber S, Barajas PC, Fried A, Liddle HA. Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems. J Consult Clin Psychol. 2008;76(4):544–55. https://doi.org/10.1037/0022-006X.76.4.544.

    Article  PubMed  PubMed Central  Google Scholar 

  65. McHugo GJ, Drake RE, Whitley R, Bond GR, Campbell K, Rapp CA, et al. Fidelity outcomes in the national implementing evidence-based practices project. Psychiatr Serv. 2007;58(10):1279–84. https://doi.org/10.1176/ps.2007.58.10.1279.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

Dr. Drahota would like to express gratitude to the AMI Community-Academic Partnership, who provided valuable insight and recommendations as the Toolkit was developed. Additionally, Dr. Drahota would like to thank Dr. Patricia Chamberlain for permitting the adaptation of the Stages of Implementation Completion. Ms. Tschida and Dr. Drahota would also like to acknowledge Dr. Steven Pierce, who provided invaluable statistical consultation.

Funding

This work was supported by NIMH K01 MH093477; PI: Drahota. Funding was used for the design and data collection of the pilot study of the ACT SMART toolkit.

Author information

Authors and Affiliations

  1. Department of Psychology, Michigan State University, 316 Physics Rd, East Lansing, MI, 48824, USA

    Jessica E. Tschida & Amy Drahota

  2. Child and Adolescent Services Research Center (CASRC), 3665 Kearny Villa Road, Suite 200N, San Diego, CA, 92123, USA

    Amy Drahota

Authors
  1. Jessica E. Tschida
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amy Drahota
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JET contributed to conceptualization, selection of methodology, formal analysis, and writing of the original draft of the present manuscript. AD conducted the pilot study of the ACT SMART Toolkit and contributed to conceptualization, selection of methodology, and review and editing of the present manuscript. Both authors read and approved the final manuscript.

Corresponding author

Correspondence to Jessica E. Tschida.

Ethics declarations

Ethics approval and consent to participate

All participants provided informed consent. The present study received exempt status from IRB review from Michigan State University to conduct secondary analysis. Data was collected while the senior author was at San Diego State University, under the IRB approval # 961087.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1. Appendix A.

ACT SMART Implementation Milestones Form. Appendix B. ACT SMART Activity Fidelity Form. Appendix C. ACT SMART Implementation Team Engagement Rating Scale.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tschida, J.E., Drahota, A. Fidelity to the ACT SMART Toolkit: an instrumental case study of implementation strategy fidelity. Implement Sci Commun 4, 52 (2023). https://doi.org/10.1186/s43058-023-00434-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s43058-023-00434-2

Keywords

Implementation Science Communications

ISSN: 2662-2211