Assessing the implementation fidelity of HPV self-collection offered by community health workers during home visits (the EMA strategy): a case study in a low-middle-resource setting in Argentina
Implementation Science Communications volume 4, Article number: 4 (2023)
In Argentina, HPV self-collection offered by community health workers was demonstrated to be effective to improve cervical cancer screening uptake. Based on these findings, the EMA strategy was scaled up in nine Argentinian provinces. However, there is no evidence about the degree of fidelity—in relation to the core components proposed by the National Program on Cervical Cancer Prevention—with which this strategy was implemented in the new jurisdictions. We carried out a fidelity evaluation of the EMA strategy scaling-up aimed at evaluating the level of adherence to the core components of the EMA strategy, and how different moderating factors affected the implementation fidelity.
This descriptive study used a multi-method approach involving quantitative and qualitative evaluations of the implementation fidelity using the Conceptual Framework for Implementation Fidelity. Evaluation of the degree of adherence to the core components of the EMA strategy was carried out through the analysis of a self-administered survey of health promoters, observations, and secondary data from the National Screening Information System. The analysis of moderating factors was carried out through analysis of field notes, and semi-structured interviews with key stakeholders.
Our results showed that the core components with highest fidelity were training, sample handling, and transportation. Regarding the offer of HPV self-collection, we found some adaptations such as locations in which health promoters offered HPV self-collection, and fewer pieces of information provided to women during the offer. In the follow-up and treatment core component, we found a reduced adherence to triage and colposcopy. Some contextual factors had a negative impact on implementation fidelity, such as urban insecurity and the reduction in the number of health promoters that offered HPV self-collection. Moderating factors that contributed to achieve high level of fidelity included a well-defined strategy with clear steps to follow, permanent feedback and high level of engagement among implementers.
Our study shows how the analysis of fidelity and adaptations of HPV self-collection in real-world contexts are key to measure and maximize its effectiveness in low-middle-income settings.
Cervical cancer is one of the leading causes of cancer death among women from low- and middle-income countries (LMIC). The disease is highly preventable with existing knowledge and technologies, and that is why the World Health Organization (WHO) has launched a global initiative to eliminate cervical cancer . One key strategy to control the disease is HPV self-collection, which is effective at increasing screening uptake, especially among hard-to-reach women who are at a higher risk of this disease [2–7].
However, for HPV self-collection to have an impact in screening coverage and disease detection at the population level, it needs to be implemented in programmatic contexts.
In Argentina, HPV self-collection was implemented in the province of Jujuy as part of the EMA study (Self-collection Modality Trial, for its initials EMA in Spanish), a mixed methods study that included a cluster-randomized controlled trial to evaluate the effectiveness of HPV self-collection offered by community health workers (CHWs) at home visits to increase screening uptake . The strategy resulted in a fourfold increase in screening uptake (from 20.2 to 85.9%), demonstrating that the strategy was effective to improve cervical cancer screening . In 2014, EMA strategy was scaled up to the whole province .Evaluation of the scaling-up in Jujuy showed that the strategy resulted in a 45% increase in screening of vulnerable population, and allowed the identification of core components of the strategy, which could be implemented in new contexts. Based on these findings, the EMA strategy was scaled to other settings, and at present, nine Argentinian provinces use it to increase screening uptake among vulnerable women. The scaling-up in Argentina was carried out by the provincial programs through joint work with the team of the National Cervical Cancer Prevention Program (NCCPP). Argentina is a federal country, and the health system organization varies in each province, which can influence on how HPV self-collection is implemented locally.
One main issue of the HPV self-collection scaling-up is the implementation fidelity, this is, the degree to which an intervention is delivered as intended . This analysis is especially relevant for complex interventions that consist of several core components (i.e., active ingredients that were essential to achieve the intended outcomes) . Otherwise, there is a risk of evaluating effects of an intervention that has been designed as part of a randomized trial, but not fully implemented when scaled up. Maximizing fidelity to core components will be highly associated with the success in achieving change in targeted outcomes and having long-term public health impact . Although there is wide evidence regarding acceptability and effectiveness of HPV self-collection in different settings, there is no evidence about the degree of fidelity with which this complex intervention is implemented in real-word contexts. To our knowledge, this is the first study to analyze fidelity of an HPV self-collection strategy using the Conceptual Framework for Implementation Fidelity (CFIF) in a low/middle-resource setting. Our results provide useful evidence about implementation fidelity during a scaling-up process that is key for countries that are considering the incorporation of HPV self-collection into their cervical cancer prevention programs.
We present the results of an evaluation of implementation fidelity of the EMA strategy scaling-up in Argentina. The study was carried out in La Matanza, a district in the Metropolitan Area of Buenos Aires, an urban setting very different from Jujuy (Fig. 1) where the EMA study was conducted. The specific aims of the study were (1) to evaluate the level of adherence to the core components of the EMA strategy, and (2) to evaluate how different moderating factors affected the implementation fidelity. The Standards for Reporting Implementation Studies (StaRI) Statement were used to guide research reporting (Additional file 1) .
We carried out a descriptive study using multiple-methods involving quantitative and qualitative evaluations of the implementation fidelity using the CFIF . A multiple-method study uses both quantitative and qualitative techniques, as distinctly separate parts of one research program . Evaluation of the degree of adherence to the core components of the EMA strategy was carried out through the analysis of a self-administered survey of health promoters, observations, and secondary data from the national screening information system (SITAM, for its initials in Spanish). The analysis of moderating factors was carried out through semi-structured interviews with key stakeholders, and analysis of field notes that provided complementary information on different factors that affected the fidelity of implementation. Phases of EMA strategy implementation and evaluation are showed in Fig. 2.
The study was carried out in La Matanza, a district in the Metropolitan Area of Buenos Aires (Fig. 1). La Matanza has 2 million inhabitants, 50% of which are poor. La Matanza’s public health system comprised a network of hospitals and primary health care centers. For the uninsured population, health services are provided free of cost, including screening, diagnosis, and treatment. In La Matanza, programmatic population-based self-collection was initiated in 2017. HPV testing (Hybrid Capture 2; Qiagen, Germantown, MD, USA) was introduced as primary screening for women aged 30 and older attending the public health system. HPV self-collection is offered by health promoters during home visits. Unlike CHWs in Jujuy, health promoters did not belong to the permanent staff of the health system; they were women from the community who received social plans (conditional income transfer for protection of families in poverty conditions) provided by the Social Development National Ministry.
Conceptual Framework for Implementation Fidelity
The CFIF  was used as a conceptual model to retrospectively evaluate the implementation fidelity of EMA strategy. We chose the CFIF because it is particularly appropriate to evaluate implementation fidelity of complex interventions—as the EMA strategy—because it allows a comprehensive assessment of different dimensions of implementation fidelity and the moderating factors that may influence it (Fig. 3). This model was integrated in all stages of the research, including conceptualization (e.g., selecting implementation components on which to focus), data collection (e.g., using components of the conceptual model to design interview guides), and analysis.
Several terms have been used interchangeably with fidelity in the literature, including treatment fidelity [14–17], intervention fidelity [18, 19], implementation fidelity [9, 10], or fidelity of implementation . This variety in terms associated with fidelity is related to the fact that initially, fidelity was mostly referred to the evaluation of what was delivered clinically in randomized trials. At present, it applies to all types of studies, from tightly controlled efficacy trials to implementation studies, although the focus on fidelity varies across each type of study . Following Carroll et al. , we have chosen the term “implementation fidelity” defined as the degree to which the EMA strategy was implemented as it was intended in the original model. It relates to the process of implementation of core strategy components . Implementation fidelity is critical to a successful translation of evidence-based strategies into real contexts, and it is a factor that may influence the relationship between the strategy and its intended outcomes. Evaluation of implementation fidelity is particularly important given the greater potential for inconsistencies in implementation of a strategy in the real world rather than in experimental conditions .
For Carroll et al. , the measurement of implementation fidelity is the measurement of adherence. Adherence includes the subcategories of content (i.e., “active ingredients”), frequency, duration, and coverage (i.e., dose). The content of the intervention may be seen as its “active ingredients” or components that are essential to achieve the indented outcomes. In our case study, the content was defined as the core components of the EMA strategy (which are described below). Coverage refers to the degree to which women who met inclusion criteria accepted the intervention, and frequency and duration refer to whether the intervention/strategy was delivered with the regularity and duration planned by its designers. In addition, the level of implementation fidelity achieved may be influenced by other variables called moderating factors. The original model described four moderating factors: intervention complexity, facilitation strategies, quality of delivery, and participant responsiveness. The CFIF model used in this study has been modified based on the adaptation made by Hasson et al. [10, 22] for the assessment of fidelity of complex interventions. Following Hasson’s model, we have included the following moderating factors:
Participant responsiveness refers to how well participants respond to or are engaged by the intervention. In our case study, responsiveness refers to the engagement of health staff involved in EMA strategy implementation.
Intervention complexity refers both to intervention and its implementation strategy characteristics—e.g., number of core components—and to the way in which they have been described and transmitted to the implementers .
Facilitation strategies refers to support strategies that may be used both to optimize and to standardize implementation fidelity, i.e., to ensure that everyone is receiving the same training with the aim of the delivery of the intervention to be as uniform as possible (e.g., manuals, monitoring and feedback) .
Context refers to the surrounding social systems such as structures and the culture of the organizations and groups, inter-organizational linkages, and historical as well as concurrent events [10, 14].
The CFIF states that different moderating factors might affect, positively or negatively, the implementation process and its fidelity. In addition, factors interact with each other, and the effect of one factor on fidelity might be influenced by another moderating factor .
Description of the EMA strategy
The EMA strategy is a complex evidence-based intervention that was implemented, evaluated and scaled up nested into the Jujuy Demonstration Project (JDP) —more details about the EMA strategy can be found elsewhere [3, 8]. It consists of several active ingredients that address multifaceted processes within interpersonal, organizational, and community contexts [24, 25]. It is based on the concept of Cancer Care Continuum defined as a process of care consisting of several steps (screening-diagnosis-treatment) and interfaces between patients, providers, and organizations. Central to the process of care across the continuum is the transfer of information and responsibility from one institution to another, from one health professional to another, and from providers to patients . The EMA strategy involves door-to-door offer of HPV self-collection to women by trained health staff together with provision of information about how to perform self-collection, sampling handling, and transportation of samples to the HPV laboratory, follow-up, and treatment of HPV-positive women at health centers (Fig. 4) . These components are essential to achieve high level of adherence to screening, triage, diagnosis, and treatment, the necessary steps to assure the screening program effectiveness to prevent cervical cancer. CFIF dimensions applied to the EMA strategy are as follows:
As mentioned above, the EMA strategy included four core components that were defined as the content of the intervention. Table 1 shows specifications for each core component following Proctor et al. model .
Self-collection should be offered by trained health promoters. Training should include at least (1) two workshops about cervical cancer prevention and HPV test, with experts’ presentations, discussions in small groups, and role playing to recreate different scenarios during the offer (Table 2), and (2) at the end of the last workshop, inviting health promoters to complete a self-administered survey to evaluate knowledge acquired during training.
Offer of self-collection
Self-collection offered during home visits by trained health promoters should include at least (1) identifying the target population, (2) offering of self-collection, (3) providing women with information about cervical cancer prevention and HPV self-collection (Table 3), (4) offering women to perform HPV self-collection, and a 10-min step-by-step explanation on how to perform it using communication support material (Additional file 2), and (5) providing women with the HPV collector to collect the sample.
Sample handling and transportation
Sample handling and transportation should include at least (1) labelling collectors with the woman’s name and the national unique identification number, (2) filling in sample collection forms, (3) transporting the samples at room temperature to primary health care centers and then to the HPV laboratory, (4) assuring that specimens arrive at the HPV laboratory within 14 days after sample collection, and (5) discarding samples without liquid, brush, or identification data at the HPV laboratory.
Follow-up and treatment
Follow-up and treatment should be organized according to national guidelines : (1) HPV-positive women should be referred to cytology triage. HPV-positive women with normal cytology should repeat the HPV test in 18 months, (2) women with abnormal cytology (ASCUS+) should be referred to colposcopy, and biopsy if needed, (3) identified cases of CIN2+ should be treated according to standard protocols (loop electrosurgical excision procedure (LEEP) or conization, and (4) HPV-negative women should be repeat screening in 5 years.
The “dose” of EMA strategy was defined based on the expected duration of the step-by-step explanation on how to perform self-collection (10 min).
Coverage was defined as the proportion of women who accepted self-collection after the offer of self-collection. The expected acceptability of self-collection was based on results of the EMA study (86%).
Indicators and sources of data used to measure adherence (content, dose, and coverage) to the core components of the EMA strategy are presented in Table 4.
Quantitative data sources
To evaluate the implementation fidelity of the core components “offer of self-collection” and “sample handling and transportation,” we conducted observations with limited participation during routine self-collection offers. Between June 14 and July 23, 2019, four female trained observers carried out 74 observations. All the health promoters of La Matanza who received training in 2017 and offered self-collection in 2019 were eligible to participate. In total, 78 health promoters who were working in the PHC system were eligible. Using a computer-generated random number list, we selected a sample of 20 health promoters to be accompanied during a workday. An observation protocol (checklist) was developed based on the competencies and activities carried out by the health promoters. We first identified a comprehensive list of planned activities by reviewing programmatic documents and training materials. We then classified this list of activities according to the content of the strategy (Table 3). The list of activities was validated by the training coordinator of the NPCCP. For each planned activity, we evaluated whether it was implemented according to the EMA model (Yes/No). In addition, observers registered any adaptation of the offer.
An ad hoc self-administered survey was developed to evaluate knowledge acquired by health promoters during the training workshops. At the end of the last workshop, health promoters were asked to complete an anonymous, self-administered survey. In total 171 health promoters completed the survey. This survey included 11 questions (multiple choice) regarding scientific data on cervical cancer and its relationship with HPV, basic information on HPV testing, the step-by-step offer of self-collection, and the follow-up of HPV+ women. Multiple choice options included only one correct answer. Results were registered in a specific database for processing and analysis.
National screening information system (SITAM)
We used data extracted from SITAM to evaluate the core component “follow-up and treatment” We analyzed the SITAM database containing records of all women aged 30 years and older screened in La Matanza using HPV self-collection during 2017–2018, and data recorded until December 2019 for the follow-up. Colposcopies, biopsies, and treatments not registered in SITAM were considered lost to follow-up. The data were accessed by authorized healthcare workers and researchers.
Qualitative data sources
During April–June 2020, we carried out six semi-structured interviews with key informants to explore their views on the EMA strategy and moderating factors that affect its implementation. We selected key informants who had been involved in the implementation of the self-collection strategy since it was introduced in 2017 and played a central role in different stages of the process. Interviews were conducted in Spanish by a female social science researcher, specialist in qualitative research (VSA). She did not live in La Matanza and did not have any relation whatsoever with the health care facilities or their authorities. Due to the COVID-19 pandemic, these interviews were carried out online. Online tools for data collection are suitable for different topics and allowed us to solve logistical issues . In addition, we analyzed field notes that were taken during observations, and training materials (e.g., training documents).
Indicators were analyzed through frequencies and percentages. The percentage of implemented activities was calculated as follows: Total number of implemented activities /total number of planned activities following EMA model*100. The percentage of implemented activities represented the degree of fidelity. Based on other studies that evaluate implementation fidelity in community settings , the following scoring categories were used in this study: 80–100%, high; 79–51%, moderate; and ≤ 50%, low.
We also calculated the percentage of adequate knowledge, defined as percentage of correct answers in the self-administered survey as follows: number of correct answers / total number of answers (adequate knowledge >70% correct answers).
Qualitative data were analyzed thematically  using the CFIF dimensions. Two researchers became independently familiar with the data through audios and transcriptions and classified data using an initial codebook, to later compare and generate themes , debate, and resolve the inconsistencies with the other team members. We sought quotation examples that adequately graphed each theme and were the most relevant to assess implementation fidelity. Atlas.Ti (version 7.5.4, ATLAS.ti Scientific Software Development GmbH, Berlin) was used for data processing.
Municipal health authorities and professionals actively participated in the planning, implementation, and fidelity evaluation of the strategy. The Director of Training of the Primary Health Care Direction of La Matanza, who was a co-researcher, participated in the design, implementation, and data analysis. All the staff of the Secretariat of Health helped to organize the field work and participated in meetings where several aspects of the project were discussed. The Secretary of Health of La Matanza, health authority in charge of the primary health care centers, gave his support to the project. In addition, NPCCP staff participated in the design, checklist elaboration, field work, and data analysis. Besides, Argentina NCI funded the study.
Adherence to the core components of EMA strategy
Training was implemented with high level of adherence. In 2017, national and municipal team members led two workshops (100% of planned workshops). These teams planned and carried out the workshops following the EMA strategy model, which included experts’ presentation, role playing, and discussions in small groups. In total, 95% of health promoters participated in both workshops. In the last workshop, acquired knowledge was evaluated, 91% of health promoters had adequate knowledge about the strategy (Table 5, Fig. 5).
Offer of self-collection
This core component was implemented with adaptations. Percentages of adherence among its 16 items were highly variable (from 22 to 82%). Fifty-four percent of self-collection offers were carried out during home visits and 46% were carried out in community health meetings and waiting rooms in primary health care centers. Regarding “Identification of target population,” most health promoters (70%) asked women their age; however, previous HPV testing and pregnancy status were asked less frequently; in some cases, health promoters evaluated the eligibility status without asking women specific information. Observations of information provided about cervical cancer prevention and HPV self-collection showed that in 68 and 82% of self-collection offers, health promoters mentioned the relationship between the test and cervical cancer prevention and the result-delivery methods, respectively. However, information about the HPV test characteristics (43%), self-collection (27%), and HPV results (41%) was provided to a lesser extent. Four out of the six items included in the “step-by-step explanation about how to perform self-collection” were mentioned in more than 54% of observations (range 54–90%). Communication support materials were used in 48% of the offers (Table 5, Fig. 6).
Sample handling and transportation
Follow-up and treatment
Between May 2017 and December 2018, 5069 women aged 30 and older were screened with self-collection (51% of total HPV-tested women, n = 5069/9977). Among them, 16% were HPV positive. Thirty-eight percent of HPV-positive women had a triage test at 120 days after screening showing a low level of adherence. Adherence to diagnosis and treatment were higher: 64% of women with positive triage performed a colposcopy. In total, 13 CIN2+ were identified and 100% were treated.
In general, health promoters spent less time than stipulated to offer self-collection (mean 8 min; range 1–17 min); therefore, as we mentioned above, fewer pieces of information were provided.
During the study, 63 of 74 visited women were considered eligible to perform self-collection. Among them, 79% accepted to perform it.
We found a high level of engagement among all actors involved in the implementation. Decision makers from the three levels (national, provincial, municipal) were involved since the beginning, with participation in round tables organized by the NPCC to plan the EMA strategy implementation at the local level. As mentioned by one of the interviewees, this involvement of all governmental level was key to integrate and coordinate all the necessary activities:
It requires a lot of will at the level of health policy. I mean, not only from the person who is carrying it out... In other words, it seems to me that this worked at least in [La] Matanza because there were people to make it work in the municipality, and in the nation and in the province, in other words, the three of them. Not from just one because just one can’t achieve this.
(E1. Training Coordinator)
Additionally, all interviewees recognized the active involvement of health professionals, HPV laboratory staff, and the health promoters. This involvement was related to the EMA strategy, which was considered an innovative and important tool to facilitate the health system to reach a population that is usually out of reach. In agreement with this recognition, data from field notes showed high acceptability and motivation among health promoters and their supervisors.
The self-collection changed everything... because sometimes many women were reluctant to go to the gynecologist. This was revolutionary, this was bringing health into their homes... so innovative...
(E3. Supervisor of Primary Health Care Direction)
... Today, I can’t imagine working on cervical cancer prevention without the HPV test. I would say it would be like taking 25 steps backwards (...) I think the HPV test and the whole [EMA] strategy to ensure women’s access is fundamental. We are fully confident that this is the way forward.
The EMA strategy was defined by the interviewees as a complex intervention due to the several core components that included and the several health system institutions and providers that needed to be involved:
I think it’s a complex process that requires many actors and each of them must carry out their part correctly, because if someone takes the sample and doesn’t store it properly or if when they go to offer it, they don’t explain it well… There are many participants, and they all have to be linked together, it’s a gear…
(E1. Municipal Training Coordinator)
[The most complex issue is] the relation with the second-level hospital, to continue with triage and treatment. But this was worked on and greatly improved.
(E3. Supervisor of Primary Health Care Direction)
In spite of its perceived complexity, stakeholders characterized the EMA strategy as well-planned. All agreed in defining the core components as clear stages to follow, and they mentioned that they had clear and sufficient information on how to proceed during the implementation. The interviewees highlighted the relevance of the training, and they described it as balanced between the use of accurate medical terms and accessible language to laypersons.
[The strategy] isn’t too complex. The key part of it is training the health personnel. Once they’re involved and trained, it’s very easy.
[The training content] was perfect according to the target audience, it was very well thought out: they didn’t talk in technicalities, they went straight to the point but not as if they were children. Everything was said with the correct terminology, but with simple and easy explanations...
(E1. Municipal Training coordinator)
A key facilitation strategy was the permanent feedback among implementers (e.g., the laboratory team, the navigator, and the supervisors). As the implementation progressed, this allowed them to identify problems in sample handling, transportation, or data registration and therefore facilitate their resolution. These activities affected positively implementation fidelity. In fact, sample handling and transportation was one of the core components with high level of fidelity (Table 5, Fig. 7).
We held a meeting with all the supervisors and health promoters on sample issues, as sometimes the samples arrived without the forms and in the meeting, we said, ‘But a key person is missing here, the driver who carries the samples is missing’. So, we met with the driver to explain.
(E4. Supervisor of Primary Health Care Direction)
In addition, training organization and education materials were also considered facilitators of the strategy: for example, the meetings held during the pre-implementation phase to plan training activities and adapt training materials to local needs, as well as the use of handbooks and flyers to guide and standardize the delivery of the intervention.
[The health promoters] received explanations on how to write the data in clear handwriting... everything was explained. They were given a flip chart with an explanation to be provided to the women, as well as handouts. They could keep the flipchart and that could help them explain everything to the women...everything they had learned....
(E1. Municipal Training Coordinator)
Some contextual factors had a direct impact on implementation fidelity. As above mentioned, health promoters did not belong to the permanent staff of the health system. This fact sometimes produced weak linkage with the formal health system and unstable or unsustainable financing. For example, financial resources for the employment of health promoters fluctuated during the implementation phase. In 2019, there was a change in the national Social Development Ministry policy of health promoters. This change resulted in a decrease of the number of health promoters, which was reduced from 171 in 2017 to 78 in 2019.
In addition, social insecurity negatively affected implementation fidelity. La Matanza is a complex urban setting. It is a large territory, with social inequality and with high levels of insecurity (i.e., perception of risk as well as lack of safety because of a high level of crime in some suburban areas). Data from semi-structure interviews and field notes showed that insecurity was an important moderator of the implementation (in fact, one of the observers had to interrupt field work due to a shooting incident).
[when we were not working in our usual neighborhood] It was very difficult to go into a house because of the safety issue. You know that not everyone opens the door of their house...
(E1. Municipal Training Coordinator)
Due to insecurity problems and the reduction of health promoters, local health staff decided to offer HPV self-collection during community activities carried out in schools, churches, fairs, and squares. This was possible because of the presence of a motivated health system staff that understood the context and that had a broad experience in primary health care and community work. In addition, the municipality of La Matanza had a health trailer that was used to provide women with a place to perform HPV self-collection when offered at community places. Interviewees evaluated this adaptation as a satisfactory response to guarantee the intervention.
In addition to the door-to-door offer... we implemented what were called ‘healthy circuits’ which is where the mobile hospital went. This was a trailer that had several clinics, where they worked on other issues such as diabetes, hypertension... And there the health promoters offered the self-collection test... That same moment, in the trailer bathroom, they would take the self-collection test.
(E3_Supervisor of Primary Health Care Direction)
This is, to our knowledge, the first study that analyzed implementation fidelity of an HPV self-collection strategy using implementation science methods. Utilization of the CFIF allowed us to carry out a comprehensive evaluation of adherence to core components of the EMA strategy and to identify key factors that influenced its implementation. Using a multi-method approach, we analyzed the implementation fidelity of the EMA strategy in La Matanza, an urban and low-middle-resource setting in Buenos Aires, a different context from Jujuy, in which EMA strategy was designed. Our results showed that the core components with highest fidelity were training and sample handling and transportation. Regarding the offer of self-collection, we found some adaptations such as the locations in which health promoters offered self-collection, and fewer pieces of information provided to women. In follow-up and treatment, we found a reduced adherence to triage. Some contextual factors had a negative impact on implementation fidelity, such as insecurity and the reduction in the number of health promoters that offered self-collection. Moderating factors that contributed to achieve high level of fidelity included a well-defined strategy with clear steps to follow, permanent feedback, and high level of engagement among implementers.
Our results showed that training was implemented with high level of fidelity. The provision of supportive materials for training may have a direct impact on the quality with which the strategy is delivered, and this may, in turn, affect the fidelity with which a strategy is implemented . Key stakeholders mentioned that training organization and education materials were facilitators of the strategy. Similar results were found in Jujuy, where CHWs mentioned that training organization was the main facilitator of the implementation of EMA strategy . Training is a key core component that allows standardization of complex interventions and should be considered as a priority core component in future scaling-up of the EMA strategy.
The core component “offer of self-collection” was implemented with adaptations: collective rather than individual offer and change of the place where it is offered. These adaptations were due to context factors: urban context, insecurity, and the increase in the number of women to be visited by each health promoter after cuts in social plans resulted in less health promoters available to offer self-collection. These adaptations emerged during the implementation process and were proposed by health staff with high motivation and engagement. This result is similar to the result reported by Hassson et al. [10, 22]. In their study they observed that staff enthusiasm for the project (responsiveness) was high, and this seemed to be a reason for adapting components to the intervention . Wide evidence illustrates how adaptation of evidence-based interventions is the rule rather than the exception when used in real-world practice . Although there are studies showing high fidelity to be related to better outcomes than low fidelity [15, 17], other studies suggest that adapted interventions may be more effective than non-adapted ones . As Carroll et al. mentioned, the evaluation of implementation fidelity provides a scope for identifying adaptability to local conditions. The adaptations observed during our case study could inform future scaling-up of EMA strategy in other urban settings with a dearth of human resources for the offer of self-collection.
Our results also showed that health promoters spent less time during the offer (dose), and consequently, the women received fewer pieces of information, especially regarding HPV testing and its role in cervical cancer prevention and results delivery and how to follow-up if positive. Our results suggest that these adaptations did not affect the acceptability of HPV self-collection among women, but they might have had an impact on adherence to triage and follow-up (a core component in which we observed low level of fidelity). In addition, not fully understanding the role of self-collection in cervical cancer prevention might reduce women capacity to make an informed decision. Strategies to support women with all the information included in the offer of self-collection core component might be needed to guarantee this, e.g., an app aimed at providing information and counseling to HPV+ women is being developed  and it might contribute to complement the explanation provided by health promoters.
HPV testing is only effective if all the women receive the corresponding follow-up and treatment. In our study, the adherence to triage was 38% at 120 days. Limited compliance to triage has been reported in Argentina [8, 23, 37, 38] and other countries [39–41]. Studies that analyzed why screen-positive women failed to complete follow-up found that in most cases it was due to problems related to the health service organization [42, 43] (i.e., delays or failures in result delivery, lack of appropriate guidance concerning the steps to follow after receiving a positive test). The EMA strategy is a complex intervention consisting of several steps that address multifaceted processes within interpersonal, organizational, and community contexts that may affect implementation fidelity. For example, if self-collection is offered during community meetings the delivery of results and referring patients for follow-up is challenging. The use of mHealth technologies to send reminders to women could be a feasible tool to reduce the time from screening to triage and improve patient-provider communication. In Jujuy, the ATICA project , an effectiveness-implementation hybrid type I trial, showed that a multi-component mHealth intervention increased adherence to triage among women with HPV+ self-collected tests. This strategy could be scaled up to improve the effectiveness of self-collection and reduce cervical cancer incidence and mortality.
Participant responsiveness refers to how well participants are engaged by an intervention/strategy [9, 22]. Several studies have shown that higher levels of implementation fidelity can be achieved when those responsible for implementing a strategy are engaged [9, 10]. In our study, participant engagement was high, with active participation of health authorities during design and implementation of self-collection, and high acceptability of the strategy by health promoters. Joint ownership and design of interventions has been pointed out as a facilitator strategy that contributes to increase motivation and engagement of health staff [45, 46]. Participant responsiveness also includes judgments by participants about the outcomes and relevance of an intervention . In our study, both stakeholders and health promoters considered self-collection as a tool that allowed the health system to reach a population that was usually unreachable and, therefore, have an impact in the burden of the disease. Feedback is also a key facilitator of participant responsiveness . In our study, permanent feedback among the laboratory team, the navigator, and the supervisors contributed to identify and solve problems related to sample handling and transportation and therefore achieve a high level of fidelity. Evidence has shown that a balanced package of incentives (both financial and non-financial) may also increase health promoters’ motivation [46, 47]. In our study, health promoters did not receive any extra financial incentive, but they were paid through a social plan. Our results are similar to those reported by Curotto et al.  that showed that the possibility of having an active role in cervical cancer prevention activities was a main motivation factor for health promoters who were paid staff of the health system.
Understanding the economic and political context in which complex interventions are implemented is key for the analysis of implementation fidelity [10, 22, 49]. The health system in which interventions take place, its general human resource policies and financing, define the space in which interventions can operate regarding incentives, working conditions, and career perspective . Unlike health promoters in Jujuy, in La Matanza, health promoters were paid through a social plan but did not belong to the permanent staff of the health system. This fact resulted in unstable or unsustainable financing, and consequently, a reduction of the number of health promoters that were involved in cervical cancer prevention. Lack of adequate financing is one of the major obstacles that is keeping health promoter programs from reaching their full potential . A recent systematic review showed that lack of financing was the most common challenge (together with the lack of supplies) facing 29 national health promoter programs in LMIC [45, 51]. In our study, lack of funding had a negative impact on fidelity because it reduced the availability of trained human resources and the frequency of the offer of self-collection”.
Incorporating research into implementation, and political will, has been identified as factors associated with faster diffusion of an innovation and its sustainability [52, 53]. Our research project was carried out in a programmatic context and run collaboratively between CEDES (an NGO with broad experience in public health research), the NPCCP, and municipal health authorities and staff. In La Matanza, stakeholders participated in the plan, implementation, and fidelity evaluation of the strategy. Thus, this study can serve as a model of how implementation science can be used to plan, conduct, and evaluate strategies implemented on a large scale in programmatic, real-world contexts. The evaluation of implementation fidelity of complex strategies in real-world contexts is crucial to identify factors that limit or facilitate their scaling-up. It is particularly important in low- and middle-resource settings, which need to optimize resources to benefit as many people as possible and improve public health.
Our study has some limitations that should be considered. Observations might bias results due to the influence that the presence of a researcher (an outsider) may produce during the observation. Thus, results regarding the offer of self-collection might not reflect how it is usually offered in the absence of the observers. Despite this limitation, observations are one of the most widely used techniques for content evaluation in fidelity studies, because they are more appropriate than interviews in which participants only self-report and allow a deep description of the implementation context [10, 30]. In our study, to minimize observation bias, we included female observers with no link with the local health system, who were trained so as not to interfere during the offering of self-collection. In addition, health promoters were explained that observations were not an evaluation of their performance, but a way to understand how self-collection was locally offered. Health promoters were also reassured about anonymity of data collected during the observations.
HPV self-collection has great potential for cervical cancer prevention, with a concrete possibility of saving thousands of lives in LMIC. Implementation science tools and frameworks are essential to enhance the way HPV-based strategies in cervical cancer prevention are implemented and evaluated. Our case study shows how HPV self-collection in programmatic contexts can be strengthened by using frameworks and tools used by Implementation Science. Specifically, it shows how the analysis of fidelity and adaptations of HPV self-collection in real-world contexts are key to measure and maximize its effectiveness in LMIC.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Atypical squamous cells of undetermined significance or worse (Include: atypical squamous cells of undetermined significance atypical cells cannot rule out high-grade squamous intraepithelial lesion (ASC-H); low/high-grade squamous intraepithelial lesions (LSIL /HSIL); and cancer)
Conceptual Framework for Implementation Fidelity
Community health workers
Jujuy demonstration project
National Cervical Cancer Prevention Program
National Screening Information System (for its initials in Spanish)
World Health Organization
Das M. WHO launches strategy to accelerate elimination of cervical cancer. Lancet Oncol. 2021;22:20–1. https://doi.org/10.1016/S1470-2045(20)30729-4.
Arbyn M, Smith SB, Temin S, Sultana F, Castle P, Collaboration on Self-Sampling and HPV Testing. Detecting cervical precancer and reaching underscreened women by using HPV testing on self samples: updated meta-analyses. BMJ. 2018;363:k4823.
Arrossi S, Thouyaret L, Herrero R, et al. Effect of self-collection of HPV DNA offered by community health workers at home visits on uptake of screening for cervical cancer (the EMA study): a population-based cluster-randomised trial. Lancet Glob Health. 2015;3(2):e85–94.
Gök M, Heideman DA, van Kemenade FJ, Berkhof J, Rozendaal L, Spruyt JW, et al. HPV testing on self-collected cervicovaginal lavage specimens as screening method for women who do not attend cervical screening: cohort study. BMJ. 2010. https://doi.org/10.1136/bmj.c1040.
Lazcano-Ponce E, Lorincz AT, Cruz-Valdez A, Salmerón J, Uribe P, Velasco-Mondragón E, et al. Self-collection of vaginal specimens for human papillomavirus testing in cervical cancer prevention (MARCH): a community-based randomised controlled trial. Lancet. 2011. https://doi.org/10.1016/S0140-6736(11)61522-5.
Arbyn M, Verdoodt F, Snijders PJ, Suonio E, Dillner L, et al. Accuracy of human papillomavirus testing on self-collected versus clinician-collected samples: a meta-analysis. Lancet Oncol. 2014. https://doi.org/10.1016/S1470-2045(13)70570-9.
Giorgi Rossi P, Marsili LM, Camilloni L, Iossa A, Lattanzi A, Sani C, et al. The effect of self-sampled HPV testing on participation to cervical cancer screening in Italy: a randomised controlled trial (ISRCTN96071600). Br J Cancer. 2011. https://doi.org/10.1038/sj.bjc.6606040.
Arrossi S, Paolino M, Thouyaret L, Laudi R, Campanera A. Evaluation of scaling-up of HPV self-collection offered by community health workers at home visits to increase screening among socially vulnerable under-screened women in Jujuy Province, Argentina. Implement Sci. 2017;12(1):1.
Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual framework for implementation fidelity. Implement Sci. 2007;2:40.
Hasson H, Blomberg S, Dunér A. Fidelity and moderating factors in complex interventions: a case study of a continuum of care program for frail elderly people in health and social care. Implement Sci. 2012;7:23.
Allen J, Linnan L, Emmons K. Fidelity and its relationship to implementation effectiveness, adaptation, and dissemination. In: Browson R, Graham C, Proctor E, editors. Dissemination and Implementation Research in Health. New York: Oxford University Press; 2012.
Pinnock H, Barwick M, Carpenter C, Eldridge S, Grandes G, Griffiths CJ, et al. Standards for Reporting Implementation Studies (StaRI) statement. BMJ. 2017;356:i6795.
Morse JM. Principles of mixed methods and multimethod research design. In: Tashakkori A, Teddie C, editors. Handbook of Mixed Methods in Social & Behavioral Research. Thousand Oaks: Sage; 2003. p. 189–208.
Hogue A, Henderson CE, Dauber S, Barajas PC, Fried A, Liddle HA. Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems. J Consult Clin Psychol. 2008;76(4):544–55. https://doi.org/10.1037/0022-006X.76.4.544.
Spillane V, Byrne MC, Byrne M, Leathem CS, O'Malley M, Cupples ME. Monitoring treatment fidelity in a randomized controlled trial of a complex intervention. J Adv Nurs. 2007;60(3):343–52. https://doi.org/10.1111/j.1365-2648.2007.04386.x.
Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS, Ory M, et al. Treatment Fidelity Workgroup of the NIH Behavior Change Consortium. Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH Behavior Change Consortium. Health Psychol. 2004;23(5):443–51. https://doi.org/10.1037/0278-6184.108.40.2063.
Cohen DJ, Crabtree BF, Etz RS, Balasubramanian BA, Donahue KE, Leviton LC, et al. Fidelity versus flexibility: translating evidence-based research into practice. Am J Prev Med. 2008;35(5 Suppl):S381–9. https://doi.org/10.1016/j.amepre.2008.08.005.
Santacroce SJ, Maccarelli LM, Grey M. Intervention fidelity. Nurs Res. 2004;53(1):63–6. https://doi.org/10.1097/00006199-200401000-00010.
Siedlecki SL. Research intervention fidelity: tips to improve internal validity of your intervention studies. Clin Nurse Spec. 2018;32(1):12–4. https://doi.org/10.1097/NUR.0000000000000342.
Dusenbury L, Brannigan R, Falco M, Hansen W. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. 2003;18:237–56.
Fixen DL, Naoon SF, Blase KA, et al. Implementation research: a synthesis of the literature. Tampa: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231); 2005.
Hasson H, Blomberg S, Dunér A. Fidelity and moderating factors in complex interventions: a case study of a continuum of care program for frail elderly people in health and social care. Implement Sci. 2012;7:23. https://doi.org/10.1186/1748-5908-7-23.
Arrossi S, Paolino M, Laudi R, Gago J, Campanera A, Marín O, et al. Programmatic human papillomavirus testing in cervical cancer prevention in the Jujuy Demonstration Project in Argentina: a population-based, before-and-after retrospective cohort study. Lancet Glob Health. 2019;7:e772–83. https://doi.org/10.1016/S2214-109X(19)30048-8.
Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, et al. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655. https://doi.org/10.1136/bmj.a1655.
Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8:139. https://doi.org/10.1186/1748-5908-8-139.
Taplin SH, Rodgers AB. Toward improving the quality of cancer care: addressing the interfaces of primary and oncology-related subspecialty care. J Natl Cancer Inst Monogr. 2010;2010(40):3–10. https://doi.org/10.1093/jncimonographs/lgq006.
Nekhlyudov L, Latosinsky S. The interface of primary and oncology specialty care: from symptoms to diagnosis. J Natl Cancer Inst Monogr. 2010;2010(40):11–7. https://doi.org/10.1093/jncimonographs/lgq001.
Arrossi S, Thouyaret L, Paul L. Recomendaciones para el tamizaje, seguimiento y tratamiento de mujeres para la prevención del cáncer cervico-uterino en el marco de la incorporación de la prueba de VPH. Buenos Aires: Ministerio de Salud de la Nación; 2015. [cited 2021 September]. Available from: https://bancos.salud.gob.ar/recurso/recomendaciones-para-el-tamizaje-seguimiento-y-tratamiento-de-mujeres-para-la-prevencion
Davies L, LeClair KL, Bagley P, et al. Face-to-face compared with online collected accounts of health and illness experiences: a scoping review. Qual Health Res. 2020;30(13):2092–102.
Pérez MC, Chandra D, Koné G, Singh R, Ridde V, Sylvestre MP, et al. Implementation fidelity and acceptability of an intervention to improve vaccination uptake and child health in rural India: a mixed methods evaluation of a pilot cluster randomized controlled trial. Implement Sci Commun. 2020;1:88. https://doi.org/10.1186/s43058-020-00077-7.
Gale NK, et al. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. 2013;13(1):117. https://doi.org/10.1186/1471-2288-13-117.
Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101. https://doi.org/10.1191/1478088706qp063oa.
Curotto M, Zalacaín-Colombo J, Paolino M, Arrossi S. Adopción e implementación del ofrecimiento de la autotoma VPH por agentes sanitarios en Jujuy, Argentina [Adoption and implementation of HPV self-collection sampling by CHWs in Jujuy, Argentina]. Salud Publica Mex. 2018;60(6):674–82Spanish. https://doi.org/10.21149/8854.
von Thiele SU, Aarons GA, Hasson H. The value equation: three complementary propositions for reconciling fidelity and adaptation in evidence-based practice implementation. BMC Health Serv Res. 2019;19(1):868. https://doi.org/10.1186/s12913-019-4668-y.
Sundell K, Beelmann A, Hasson H, von Thiele SU. Novel programs, international adoptions, or contextual adaptations? Meta-analytical results from German and Swedish Intervention Research. J Clin Child Adolesc Psychol. 2016;45(6):784–96. https://doi.org/10.1080/15374416.2015.1020540.
Sanchez Antelo V, Szwarc L, Paolino M, Saimovici D, Massaccesi S, Viswanath K, Arrossi S. A Counseling Mobile App to Reduce the Psychosocial Impact of Human Papillomavirus Testing: Formative Research Using a User-Centered Design Approach in a Low-Middle-Income Setting in Argentina. JMIR Form Res. 2022;6(1):e32610. https://doi.org/10.2196/32610.
Paolino M, Campanera A, Martiarena S, et al. Adherencia al triaje en contexto de tamizaje con autotoma del test de virus del papiloma humano en la provincia de. Jujuy Rev Argent Salud Pública. 2019;10(40):7–13 Registro RENIS No: IS001709.
Paolino M, Gago J, Pera AL, Cinto O, Thouyaret L, Arrossi S. Adherence to triage among women with HPV-positive self-collection: a study in a middle-low income population in Argentina. Ecancermedicalscience. 2020;14:1138. https://doi.org/10.3332/ecancer.2020.1138.
Sancho-Garnier H, Tamalet C, Halfon P, et al. HPV self-sampling or the Pap-smear: a randomized study among cervical screen- ing non-attenders from lower socioeconomic groups in France. Int J Cancer. 2013;133(11):2681–7. https://doi.org/10.1002/ijc.28283 PMID: 23712523.
Holme F, Maldonado F, Martinez-Granera OB, et al. HPV-based cervical cancer screening in Nicaragua: from testing to treat- ment. BMC Public Health. 2020;20(1):495.
Giorgi Rossi P, Fortunato C, Barbarino P, et al. Self-sampling to increase participation in cervical cancer screening: an RCT comparing home mailing, distribution in pharmacies, and recall letter. Br J Cancer. 2015;112(4):667–75.
Paolino M, Sankaranarayanan R, Arrossi S. Determinantes sociales del abandono del diagnóstico y el tratamiento de mujeres con Papanicolaou anormal en Buenos Aires. Argentina Rev Pan de Salud Pública. 2013;34(6):437–45.
Zapka J, Taplin S, Anhang Price R, et al. Factors in quality care—the case of follow-up to abnormal cancer screening tests— problems in the steps and interfaces of care. J Natl Cancer Inst Monogr. 2010;40:58–71. https://doi.org/10.1093/jncimonographs/lgq009.
Arrossi S, Paolino M, Sanchez Antelo V, Thouyaret L, Kohler E, Cuberli M, et al. Effectiveness of an mHealth intervention to increase adherence to triage of HPV DNA positive women who have performed self-collection (the ATICA study): a hybrid type I cluster randomized effectiveness-implementation trial. Lancet Reg HealthAmericas. 2022;9. https://doi.org/10.1016/j.lana.2022.100199.
Perry H, Crigler L, Kok M, Ballard M, Musoke D, LeBan K, et al. Community Health Worker Programs at the Dawn of a New Era: 11. Leading the Way to Health for All. BMC. Health Res Policy Syst. 2021. https://doi.org/10.1186/s12961-021-00755-5.
Naimoli JF, Perry HB, Townsend JW, Frymus DE, McCaffery JA. Strategic partnering to improve community health worker programming and performance: features of a community-health system integrated approach. Hum Resour Health. 2015;13:46. https://doi.org/10.1186/s12960-015-0041-3.
Colvin CJ, Hodgins S, Perry HB. Community health workers at the dawn of a new era: 8. Incentives and remuneration. Health Res Policy Syst. 2021;19(Suppl 3):106. https://doi.org/10.1186/s12961-021-00750-w.
Curotto M, Barletta P, Paolino M, Arrossi S. La perspectiva de los agentes sanitarios sobre la incorporación programática de la autotoma del test de VPH. Cad Saude Publica. 2017;33(4):e00138515.Spanish. https://doi.org/10.1590/0102-311X00138515.
Kok MC, Kane SS, Tulloch O, Ormel H, Theobald S, Dieleman M, et al. How does context influence performance of community health workers in low- and middle-income countries? Evidence from the literature. Health Res Policy Syst. 2015;13:13. https://doi.org/10.1186/s12961-015-0001-3.
Masis L, Gichaga A, Zerayacob T, Lu C, Perry HB. Community health workers at the dawn of a new era: 4. Programme financing. Health Res Policy Syst. 2021;19(Suppl 3):107. https://doi.org/10.1186/s12961-021-00751-9 PMID: 34641893; PMCID: PMC8506106.
Health for the People: National Community Health Programs from Afghanistan to Zimbabwe. https://chwcentral.org/wp-content/uploads/2021/11/Health_for_the_People_Natl_Case%20Studies_Oct2021.pdf. Accessed 23 Sep 2022.
Khresheh R, Lesley B. Practice–research engagement. Jordanian experience in three Ministry of Health hospitals. Action Res. 2007. https://doi.org/10.1177/1476750307077313.
Yamey G. Scaling up global health interventions: a proposed framework for success. PLoS Med. 2011. https://doi.org/10.1371/journal.pmed.1001049.
The study was funded by the by National Cancer Institute (Argentina) - Asistencia Financiera a Proyectos de Investigación en Cáncer de Origen Nacional IV. The authors would like to thank all the staff of the Secretariat of Public Health of La Matanza—especially the team Training of the Primary Health Care Direction, Primary Health Care Supervisors, and professionals of Primary Health Care Centers—for their participation in this project. We also thank municipal Health authorities for their support during the project. Finally, we especially thank the women and health promoters who participated in the study.
Research for this publication was funded by National Cancer Institute (Argentina) - Asistencia Financiera a Proyectos de Investigación en Cáncer de Origen Nacional IV.
Ethics approval and consent to participate
This protocol was approved by the COMUBI Review Board. A written informed consent was obtained from all participants in the study.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Paolino, M., Sánchez Antelo, V., Cuberli, M. et al. Assessing the implementation fidelity of HPV self-collection offered by community health workers during home visits (the EMA strategy): a case study in a low-middle-resource setting in Argentina. Implement Sci Commun 4, 4 (2023). https://doi.org/10.1186/s43058-022-00367-2