Skip to main content

De-implementation and substitution of clinical care processes: stakeholder perspectives on the transition to primary human papillomavirus (HPV) testing for cervical cancer screening



New cervical cancer screening guidelines recommend primary human papillomavirus (HPV) testing for women age 30–65 years. Healthcare organizations are preparing to de-implement the previous recommended strategies of Pap testing or co-testing (Pap plus HPV test) and substitute primary HPV testing. However, there may be significant challenges to the replacement of this entrenched clinical practice, even with an evidence-based substitution. We sought to identify stakeholder-perceived barriers and facilitators to this substitution within a large healthcare system, Kaiser Permanente Southern California.


We conducted semi-structured qualitative interviews with clinician, administrative, and patient stakeholders regarding (a) acceptability and feasibility of the planned substitution; (b) perceptions of barriers and facilitators, with an emphasis on those related to the de-implementation/implementation cycle of substitution; and (c) perceived readiness to change. Our interview guide was informed by the Consolidated Framework for Implementation Research (CFIR). Using a team coding approach, we developed an initial coding structure refined during iterative analysis; the data were subsequently organized thematically into domains, key themes, and sub-themes using thematic analysis, followed by framework analysis informed by CFIR.


We conducted 23 interviews: 5 patient and 18 clinical/administrative. Clinicians perceived that patients feel more tests equals better care, and clinicians and patients expressed fear of missed cancers (“…it’ll be more challenging convincing the patient that only one test is…good enough to detect cancer.”). Patients perceived practice changes resulting in “less care” are driven by the desire to cut costs. In contrast, clinicians/administrators viewed changing from two tests to one as acceptable and a workflow efficiency (“…It’s very easy and half the work.”). Stakeholder-recommended strategies included focusing on the increased efficacy of primary HPV testing and developing clinician talking points incorporating national guidelines to assuage “cost-cutting” fears.


Substitution to replace an entrenched clinical practice is complex. Leveraging available facilitators is key to ease the process for clinical and administrative stakeholders—e.g., emphasizing the efficiency of going from two tests to one. Identifying and addressing clinician and patient fears regarding cost-cutting and perceived poorer quality of care is critical for substitution. Multicomponent and multilevel strategies for engagement and education will be required.

Trial registration, #NCT04371887

Peer Review reports


National organizations including the United States Preventive Services Task Force (USPSTF) are recommending a relatively new approach for cervical cancer screening for women aged 30–65 years known as primary human papillomavirus (HPV) testing [1]. The use of primary HPV testing is also endorsed by the American College of Obstetrics and Gynecology (ACOG) [2]. Studies suggest primary HPV testing is superior to cytology alone (Papanicolaou testing (Pap)) [3] and is as effective as co-testing (Pap plus HPV test) [4,5,6,7]. As a result, US healthcare organizations are preparing to de-implement the previous recommended strategies of co-testing or Pap alone and initiate primary HPV testing. While few other countries have fully implemented primary HPV testing, it is strongly recommended in European guidelines and many countries are working towards full implementation [8]. Both the Netherlands and Turkey have transitioned to primary HPV testing, as well as Australia [9, 10]. Based on these experiences and modeling studies, the transition to primary HPV testing is expected to result in improved cost-effectiveness and simplified clinical care processes for those converting from co-testing to primary HPV testing (e.g., one sample taken instead of two at point of care) [11, 12]; primary HPV testing may also lead to better patient outcomes and will align with the new guidelines and associated quality metrics. These may be powerful facilitators for change. However, evidence is lacking regarding effective strategies for this type of dual-practice change incorporating de-implementation plus substitution with a new recommended practice.

Substitution is a relatively new concept and research focus in the implementation and improvement science fields. Recent scholarly and clinical efforts focusing on de-implementation have concentrated heavily on the cessation of contraindicated practices and/or overuse of practices that cause more harm than benefit [13,14,15,16,17]. In the field of cancer prevention and control, overuse is common and includes overuse of screening services, overtreatment, and overtesting during surveillance [18,19,20,21,22]. A recent editorial on de-implementation in cancer care distinguished between four types of cancer-related practices to be considered for de-implementation: ineffective, contradicted, mixed, and untested [18]. In general, these practices do not have an available substitute, and research and quality improvement efforts have focused on discontinuation or reduction in frequency. For example, the change in the USPSTF recommended initiation of breast cancer screening for low-to-average risk women from age 40 to age 50 represents a discontinuation of screening without substitute; similarly, the American Society of Clinical Oncology recommends that asymptomatic post-treatment breast cancer patients do not need surveillance imaging or laboratory tests. An example of the reduction in frequency is the evolving guideline recommendation to extend the intervals between cervical cancer screenings for low-to-average risk women. Many current de-implementation efforts in cancer care and other clinical disciplines stem from the Choosing Wisely campaign from the American Board of Internal Medicine, which focuses on clinical practices that could be reduced or completely ceased across a variety of clinical domains [13, 23,24,25,26].

In contrast to de-implementation, substitution has been defined as the promotion of one or more alternatives to an ineffective or less effective practice, in which the substitutes replace or displace the previous practice [27,28,29,30]. Ideally, to ease cognitive pressures and potentially simplify change management strategies, a substitution will have a related replacement that is a similar, closely related but more effective intervention [14, 28]. Substitution can be seen as “coupled” implementation and de-implementation [28, 29] and as a potential strategy to facilitate practice change. While implementation of novel practices is certainly challenging, de-implementation of established, routine practices is also challenging [15, 24, 31, 32]. It is possible that a coupled approach, if available, could be more favorably received by clinicians and health systems [14, 28, 29]. A coupled strategy would require a multicomponent intervention that encompasses both de-implementation and implementation goals, requiring strategies to address cessation of the existing practice and replacement with the newer, more effective practice [28].

Within the USA, the decision to transition to primary HPV screening is mainly left to individual health systems rather than implemented on a national or state level. The decision of health insurers (governmental and private) regarding coverage may influence uptake. Kaiser Permanente Southern California (KPSC), a large integrated health system (hospitals, medical groups, and insurance plan) transitioned from HPV co-testing to primary HPV testing in July 2020 in response to the updated USPSTF guidelines. As part of a multi-year study evaluating implementation strategies to facilitate HPV primary testing, we conducted a series of semi-structured qualitative interviews with KPSC members, clinicians, and administrators. Interviews were conducted prior to the practice change and any related educational efforts (e.g., clinician webinars) to gain insight into perceived barriers and recommendations for change. The overarching goal of the main study is to compare implementation strategies (tailored versus centralized strategies) to facilitate uptake of primary HPV testing. Interviews with clinicians and administrators elicited their views on (a) the acceptability and feasibility of the planned substitution; (b) their perceptions of potential barriers and facilitators, with an emphasis on those related to the de-implementation/implementation cycle of substitution; and (c) perceived readiness to change to primary HPV screening. We also explored potential strategies or recommendations to overcome potential multilevel challenges (patient-, clinician-, and system-level). Patient interviews focused on overall reactions to the proposed change and attitudes regarding this new approach to cervical cancer screening.


Study design

We conducted a qualitative study using semi-structured interviews; our findings are reported using the criteria for reporting qualitative research (COREQ). The COREQ checklist can be found in Additional file 1.

Study setting and participants

KPSC is an integrated delivery system serving over 4.4 million members. Members are racially, ethnically, and socioeconomically diverse and broadly representative of the underlying Southern California population [33]. We conducted interviews with clinicians and administrators between April and November 2019 and with patients from July 2019 to November 2019. The following types of clinical and administrative stakeholder participants were invited to participate: (1) Family Medicine, Internal Medicine, and Obstetrician-Gynecology physicians; (2) department administrators (DAs) and assistant department administrators (ADAs); and (3) nurses including medical assistants (MAs) and licensed vocational nurses (LVNs). For patients, we invited individuals who were female, aged 30–65 years, who received cervical cancer screening within the last 3 months and were English speakers. We obtained verbal consent from all participants, which was approved by the KPSC Institutional Review Board [IRB #12015], and a written informed consent from the patients for audiotaping and transcribing.

Interview guide and conceptual model

Interviews with both clinical/administrative and patient participants were facilitated using semi-structured interview guides. The guides included open-ended questions based on central research questions and probing questions, which allowed for deep exploration of participant responses and probes for emergent themes. Our iterative interview guide development was informed by the Consolidated Framework for Implementation Research (CFIR) [34]. The CFIR is widely recognized to provide a comprehensive overview of relevant constructs applicable to implementation research and clinical practice change [35]. We developed potential interview questions based on the HPV testing and implementation science literature and mapped the question domains to relevant CFIR constructs (see Additional file 1). We convened our study’s patient advisory group (3 KPSC patient members, 1 non-KPSC patient) to inform the development of our interview guides (i.e., Were we missing any key questions? Did we need to revise the interview question flow?); this group also provided input on the development of a patient-, clinician-, and system-level list of potential implementation barriers—designed with our clinical stakeholder group—to be used during the interviews with clinical and administrative participants (see Additional file 2). The interview guide was further discussed and refined by the study team after the initial interviews.

Domains in the clinician/administrative guide included (a) awareness, knowledge, and beliefs related to the practice change; (b) potential barriers to de-implementing current co-testing practices and substituting primary HPV screening; (c) perceived patient reactions to the substitution and related needs and resources; (d) site-specific culture and contextual issues with potential to impact substitution (i.e., culture, leadership, readiness-for-change); (e) clinician self-efficacy; (f) recommendations to facilitate change; and (g) champions or key leadership available at their site. Topics covered during patient interviews included (a) perceptions of guidelines and medical evidence; (b) facilitators and barriers to de-implementation and substitution, and overall acceptance of primary HPV testing; (c) identification of patient sub-populations experiencing potentially greater barriers to acceptance of primary HPV testing; and (d) strategies for increasing acceptance of this substitution as well as prioritization of strategies.


To attain theoretical data saturation, we planned 20–25 total interviews. We emailed invitations and three reminders to 238 physicians, 82 administrators, and 400 patients, assuming a 10–15% response rate based on our prior experience in conducting qualitative research with these stakeholder groups [21, 36, 37]. No prior relationships existed between the interviewers (CMP, DA) and participants prior to study commencement. Participants were informed about the interviewers’ role as qualitative researchers on the study team, who had an interest in improving cervical cancer screening practices.

Interview procedures

We conducted one-time individual semi-structured interviews. Clinician and administrator interviews were conducted in person or over the phone in a private location within the medical center by CMP, a female qualitative researcher with over 25 years of experience, and DEA, a female qualitative researcher with over 10 years of experience. Patient interviews were conducted by phone to minimize travel burden. Interviews lasted approximately 30–60 min and were recorded and transcribed verbatim into written transcripts as preparation for coding and analysis using NVivo qualitative analytical software (© QSR International 2020). We halted further interviews once we were confident that we reached thematic saturation with our interview sample whereby we were no longer eliciting new pertinent information or themes from additional interviews [38,39,40,41]. Transcription was conducted by an institutionally approved vendor. We collected age, gender, race/ethnicity, and profession data from KPSC’s electronic health records and/or participant self-report.

Coding and analysis

We conducted a two-step analysis with initial thematic analysis, followed by framework analysis guided by CFIR. This two-step process allows for an initial free-form analysis with exploration of convergent and divergent findings, followed by a mapping process to the overarching CFIR framework without the possibility of missing divergent/discordant findings. We developed an initial coding tree structure which we refined during the analysis process. To help ensure rigor and transparency, all coding development steps were tracked and reported in a codebook [42]. Consensus on the coding structure was achieved through cyclical testing of structure development by three members of the research team using a team coding approach to review and resolve discrepancies. We identified a lead coder (CMP) and two secondary coders (EEH and CH). Coders independently coded a random sample of the transcripts (n = 4), then met repeatedly to compare coding applications, discuss analytic insights and annotations, and identify and resolve discrepancies. Once consensus was achieved and discrepancies resolved, the codebook was finalized and the lead coder continued to code the remaining transcripts. Once coding was complete, the team met as needed to determine the hierarchy of themes and produced an aggregated summary of the findings categorized into domains, themes, and sub-themes. We categorized recommendations and suggestions to address barriers with the goals of (1) informing the design of the process-change activities across the organization and (2) understanding cross-cutting themes across stakeholder roles. Transcripts and data analysis were not returned to participants. While brief notes were used in post-interview summaries shared with the study team, traditional field notes were not included as part of the formal analysis.


Sample characteristics

Of the 238 clinician invitations, 26 (11%) responded yes to the interview request and 14 completed interviews. For administrators, 11 (14%) of the invited 82 responded yes, and 4 completed interviews. For patients, 10 (3%) of the invited 400 responded yes, and 5 completed interviews. We conducted a total of 23 interviews, n = 18 clinician/administrator and n = 5 patient. Table 1 provides a breakdown of the stakeholder roles and characteristics.

Table 1 Qualitative interview participant characteristics

Thematic findings

Clinician resistance to substitution

Participants perceived that some clinicians, especially in primary care, may be more resistant to this practice change due to the dual challenges of the substitution, encompassing both de-implementation (co-testing) and implementation (primary HPV testing). One physician referenced colleagues who are still struggling to adapt to older guideline changes with regard to cervical cancer screening frequency:

I’m going to be very honest with you…there are still some providers who have that old practice of wanting to do [Pap tests] every year. So…CDC guidelines are the three years. We’ll just use the three years as the minimum. There are still providers who insist on doing it every year, so we have…the over testing.

The clinical stakeholders most reluctant to accept the practice change may be members of the nursing team who have their own attachment to the Pap test/co-testing based on their personal screening experience and knowledge. A nurse worried, “I didn’t know they were going to eliminate Pap, because I’ve seen Pap [be] abnormal in the past and not be HPV ...I always worry are we making sure we are capturing everything?,” while a family medicine physician perceived that “…most of our nursing staff are women, so they themselves might have some personal bias for or against Pap changes. I think you would have to make sure that there is buy in….”

Furthermore, participants shared concerns that patient resistance and/or confusion will require clinicians to engage in longer conversations with “anxious” women:

Patients who just want to come in and patients who are unfamiliar with it will want specifically a Pap. That, again, takes time now from the physicians to convince or inform the patient, you don’t need that. You don’t need X, Y, and Z. Here’s the research, here’s the reasons why. And having those kind of like continued conversations by…patients [asking] ‘Why didn’t I get a…Pap smear?’

Highlighting that not all clinicians have the skills to communicate information about practice changes to their patients effectively, participants suggested “…it’s also [about] educating the physician” and “…[making] sure our physicians are…well-informed about how we communicate this to patients on a mass level, so that they don’t come in expecting…something else….”

Patient attachment to the Pap

Women’s attachment to the Pap (especially older women) was identified across participants as a critical challenge to the substitution. Clinicians and department administrators noted that women do not always respond to population-level statistics, which can feel impersonal, citing that many patients still demand a yearly Pap instead of the recommended interval of co-testing regardless of how much the evidence is explained to them (“…still, some of my older patients, they’re like, ‘I don’t feel comfortable not having a Pap smear every year.’”). One physician who has a high proportion of Hispanic patients noticed that they often travel to Mexico to get yearly Pap tests, while another fears “…patients will continue to request certain [tests] because we already currently have…some patients who want over screening.” Patients raised the same concerns. One woman said that it was a good thing she learned about this potential practice change in advance through her participation in the interview, because “…If I didn’t know I’d be surprised and I’d be distrustful, why did you take my [Pap] test away?”.

Fear of missed cancers/more care is better care

While many participants framed it as an attachment to the Pap test specifically, a deeper analysis suggests the fundamental basis of the concern about the substitution of primary HPV testing in place of co-testing centers around the fear of potentially missing cancer. A patient shared her concerns: “…just to make sure nothing is going to be missed, just assuring patients that this test is going to be able to find -- detect X, Y, and Z just like the Pap….” Some clinicians (particularly nurses) also expressed concern about missed cancers:

…I’ve seen Pap [be] abnormal in the past and not be HPV…Trust me, if you’re down there, what’s the difference if you take one more scraping of something? I don’t understand why they would discontinue one, take one type of abnormal cell and not go, oh, because I’ve seen them both come back abnormal at different times. I don’t understand the logic. I guess somebody would have to explain it to me, we want to eliminate it because of this amount of chance, or this percentage. But, again, if it’s your family member, do you want to be that percentage of only five percent will end up with this particular kind of cancer?

Interviews with both patients and clinicians strongly suggest that some patients hold fast to the idea that the more tests and procedures they get, the better the quality of their care. The challenge from the clinician perspective is “…convincing the patient that only one test is needed and good enough to prevent and to help detect cancer.”

Patient data suggest this concern is justified, as they question if “The standard of care right now [for women’s health] is kind of going down because the research being done is saying, ‘We don't need to do these things as often…’” and “…if we're getting rid of the regular Pap smear - is this HPV test going to be screening for all the rest of it that the regular Pap smear also screened for?” Clinicians shared numerous stories of difficult conversations with patients about changing practices to get them to understand the concept of de-implementation (e.g., the cessation of healthcare services that do not provide benefit) and that receipt of more services does not always result in better health outcomes and may even lead to harms, including unnecessary tests and procedures.

Cost-cutting and competition

Clinicians, administrators, and patients repeatedly mentioned a perception among patients (and some clinicians) that de-implementation and/or substitution efforts resulting in fewer tests and services are fundamentally driven by the desire to cut costs and/or make procedures more convenient for the clinical team and healthcare system, rather than being driven by quality and new evidence. Echoing the sentiment of many participants, a family medicine physician lamented:

…[you have to] sit and educate the patient…Because it’s always if you don’t explain it adequately that the patient somehow feels they’re being slighted, or they’re not being provided care. That we’re somehow, ‘Oh, we’re doing this to save money.’ It’s like, no, we’re not, we’re doing this because this is what evidence is seeing.

Some patients have certain expectations because of the co-pays required for their visits and procedures: “…there are patients that say, ‘I always feel like [KPSC]…is just cutting stuff out and they’re already taking my money…‘Why aren’t they just doing the Pap every year?’” In addition, clinicians shared their concerns about external perceptions if their system moves to a practice that is different from local competitors because such a shift may influence patient perceptions regarding the system’s motivation to initiate the change (e.g., will it fuel patients’ suspicions about organizational motives, thereby hurting the organization’s reputation and standing in the community?). Noting that patients come into Kaiser Permanente from other healthcare systems, or hear about the services and procedures their friends and/or family receive from other health organizations, one DA (ob-gyn) said:

Whatever is the community standard…it is more difficult for us to now say we are going to do this versus [other healthcare systems] who are in our backyards, that they will continue to do PAP smears with HPV and then we’re the odd people out…[patients then say] ‘Okay, well I get more tests [from other systems] and that is better care.’

Even some clinicians questioned the motivation for the change (“The most important thing is quality, second thing is cost. I just want to make sure we are not putting cost above quality.”) along with patients (“Yeah, I mean, I’m definitely not just this healthcare is a conspiracy [patient] or anything like that, but...I know that there’s corners cut …I would be wondering if that's what’s going on here, too.”).

Impact on access and workload

Clinical and administrative participants questioned whether the change to primary HPV testing and/or the new follow-up algorithm will result in a higher volume of patient questions and inquiries, either via increased email messaging, telephone calls, or lengthier visits (“Yeah, this is always a worry for primary care, that we’re going to get more questions or emails or it’s going to increase our indirect work…”). These participants also raised concerns as to whether the change will exacerbate pre-existing access problems in primary care and/or ob-gyn, “…Because if that’s a thing that we have to have patients come back [more]? I don’t think we have the capacity…That’s always the biggest problem for us, is access.”

While a minor theme, clinicians expressed concerns that there may be a steep learning curve to understand the new follow-up algorithm for abnormal primary HPV results. Others felt this challenge could be easily addressed with clear education and communication, “…a lot of the people here are just used to doing the cytology [Pap] and HPV. So, they know the protocol. But then, it’s a little bit different with the primary HPV but…you’ve just got to read a little bit about it.” However, an ob/gyn physician expressed a bit more concern about the learning curve for her colleagues in primary care:

I’m wondering if they’re going to completely understand in family medicine. And are they going to be able to explain to their patients correctly without saying, ‘Well, you know what? Just go be seen in Ob/Gyn.’

Implementation facilitators

Substitution with replacement: feasible and acceptable

For many clinicians and administrators, this substitution with replacement is viewed as relatively minor and an overall workflow efficiency. One DA commented, In my mind, as an operational leader, what satisfies the patient’s needs, provides the same level of care, and also makes the work of my staff and my team easier, I’m all for.” Participants in both clinical and administrative roles reiterated that they believe the change will make specimen collection easier (“I don’t think it’s going to be much of a challenge or a barrier to get the workflow started…. It’s a very easy and half the work.”) and nurses will appreciate having less to do to prepare the specimen collection trays (“Okay, one less thing we have to set up. That’s great.”). However, while a primary care physician also found the proposed practice change to be acceptable and feasible (“…if we’re just going to switch to that because we know [HPV] is the cause, it seems pretty reasonable.”), she added an important caveat mentioned by other participants as well, “I just think as long as we educate both the physicians and then the patients secondarily, it should be okay.” Another DA highlighted that a workflow change that is viewed as an efficiency can be leveraged for buy-in: “If it’s something that…reduces the workload of the providers or the staff, then generally it’s going to be accepted and adopted quicker.” In addition, based on the current evidence, the change is perceived as likely to result in fewer unnecessary and invasive tests over time. (Maybe doing less invasive procedures. You can avoid that.”)

Evidence to support substitution

Clinical and administrative participants reported having an expected threshold for the level of evidence they believe will be necessary to secure buy-in for the planned practice change to primary HPV testing (“With the support of the education, it will not be difficult.”). The level of evidence desired is described as basic, brief, and from respected external specialty associations and organizations (i.e., ACOG and the USPSTF).

Medical center culture

Descriptions of high functioning clinic culture with adaptability regarding de-implementation, implementation, and/or substitution were linked in particular to strong leadership by the DAs, because clinicians often look to them to set a tone of positivity in the clinic and lead practice change. One DA explained, “You have to build relationships with your staff. Your staff needs to trust you. And when [when] that time comes [for change], they are accommodating to you.” In addition, teams often described having facilitated reasonably smooth implementation of similar practice changes (“…we’re a pretty supportive group…It’s pretty easy. We haven’t had any major issues when we’ve had changes…”). Finally, most clinics already have identified “champions” or clinicians with subject expertise that can be leveraged (i.e., cervical dysplasia champions, primary care physicians with an interest in women’s health issues, etc.):

…we have a lot of that here, and each physician sometimes has special interests or things that they’re good at and…everybody knows, I’m the women’s health person…And then we have another guy that’s really into communication with patients and difficult patient encounters…we try to…influence each other, and we know who to go to when we have questions.

Alignment with CFIR

Viewing our findings through the lens of the CFIR framework is informative. Guided by the CFIR Intervention Characteristics domain, we asked stakeholders to reflect on the strength, complexity, and design of the planned practice change intervention. We discovered that stakeholder perceptions of acceptability (Intervention Characteristics) are high in terms of the workflow impact on clinical teams, suggesting a key facilitator that can be leveraged for successful substitution. Perceptions that the change may result in fewer unnecessary and invasive tests per the updated evidence in the new guideline fed into opinions about the acceptability of the practice change as well. However, participants repeatedly pointed out that the acceptability from the patient perspective may be lower, at least for an important minority of patients, and is a barrier that should be anticipated and met proactively. Under the CFIR domain Individual Characteristics, physician confidence was also reportedly high in terms of their belief that they and their colleagues could easily transition to the new cervical screening practice. Suggested implementation facilitators related to positive team culture, strong DA leadership, successful implementation of previous workflow changes, and an existing network of subject matter experts and physician champions fall into the CFIR domain Process & Inner Setting and also indicate a reasonably high feasibility for implementation success in the practice setting.

System-level barriers, including the impact on access and workload, as well as uncertainty about follow-up testing algorithms and patient transitions between primary care and ob-gyn fall into the CFIR domain Intervention Characteristics & Outer Setting. Potential clinician resistance and apprehension about the substitution falls into Characteristic of Individuals and should be anticipated and addressed proactively. Similarly, identified patient concerns (e.g., what happened to the PAP test?), fears (e.g., missed cancers), and potential for resistance align best with CFIR’s Patient Needs & Resources domain and were perceived by stakeholders as the most critical barriers to address.


Overall, our results demonstrate the complex nature of substitution with replacement and the de-implementation/implementation cycle of practice change. On one hand, clinicians and administrators indicated that this change in cervical cancer screening is viewed as relatively minor and a workflow efficiency. De-implementing a two-test process (co-testing with both HPV and Pap) and offering an acceptable, simpler substitution option (primary HPV test only) is a strong facilitator for this practice change. Participants recommended leveraging this perception—e.g., the substitution will make staff jobs easier, not harder—to secure buy-in from clinicians and administrators in advance of the practice change rollout. On the other hand, patients and some clinicians expressed strong fears of potential missed cancers, questioned the reason for the change, and perceived that de-implementing Pap testing was driven by cost-cutting and would result in lower value of care for women. Relatedly, we observed a common belief that more care equals better quality of care. Fears regarding cost-cutting and “more is better” are unique facets of de-implementation and/or substitution that will need to be addressed to facilitate successful practice change. This may be of particular concern in the USA, which has a competitive healthcare market [43] where patients and health insurance purchasers may select and change clinicians and/or health systems based on costs, coverage, and preferences. Typically, implementation efforts and strategies focus on the adoption of novel technologies or treatments, rather than de-implementation of low-value services [44,45,46,47]; the addition of domains specific to patient and clinician perceptions of cost-cutting, healthcare competition, and more tests and services equaling better care could strengthen existing practice change frameworks [48,49,50]. Participants suggested clinicians should be sufficiently prepared to have difficult conversations with anxious patients about the substitution. Additionally, participants recommended the development of clinician scripts and talking points and/or physician training on how to effectively discuss evidence and high-value care with patients (e.g., introducing the concept of de-implementation in relation to the reality of shifting medical evidence) [27, 32].

Participants offered other suggestions for multicomponent strategies to overcome potential system-, clinician-, and patient-level barriers. These suggestions include developing and delivering brief, concise clinician- and patient-facing education materials and resources to secure buy-in from clinicians and assuage patient fears that primary HPV testing will leave them vulnerable to missed cancers. In both cases, participants suggest using multiple channels for education both prior to and during the substitution processes. In the case of clinicians, specific suggestions included departmental announcements via email and at meetings (from departmental Chiefs of Service and DAs), in-service trainings from internal/external subject experts or “champions,” stories of positive patient impact, and independent online learning modules. For patients, specific suggestions for resource development included posters in exam rooms and communal areas within medical centers, videos, media/social media campaigns, information on the patient portal, personalized letters, and targeted reminders tied to the visit for the procedure, with messaging focused on the increased sensitivity of primary HPV screening for detecting and preventing cervical cancer and incorporating national guidelines to assuage the “more care is better” and “cost-cutting” fears. Specific recommendations to address identified barriers are presented in Table 2 and summarized briefly below.

Table 2 Participant suggestions for cervical cancer screening practice change

Barrier: patient perceptions of “lower quality” care

Suggested solution(s)

Create clinician-targeted communication scripts and/or training on how to effectively discuss the practice change and the concept of high-value care with patients.

Accompany the script with external resources from national organizations with credibility and standing (ACOG, USPSTF) to explain the change; this addresses concerns that health systems are cutting corners or not providing the best care.

Barrier: patient resistance to Pap de-implementation

Suggested solution(s)

Recommendations included developing a wide range of patient-facing resources for distribution both prior to and during the practice change implementation. A key focus should be assuaging patient concerns about “missed cancers” with current evidence of primary HPV testing efficacy and explaining concepts of guideline-recommended de-implementation of unnecessary tests and/or procedures.

Barrier: patient expectations for communication

Suggested solution(s)

Patients will likely expect proactive communication about this substitution from their physicians such as letters, emails, or in-person conversation. Participants warned that skipping patient education and outreach, a critical step in effective communication, will likely lead to longer patient visits, greater patient confusion, and mistrust of the reasons for the change (i.e., the organization is just trying to save money).

Barrier: clinician resistance

Suggested solution(s)

Provide education by presenting the evidence concisely in multiple formats (e.g., webinar, departmental meetings, email communication from leadership) prior to the rollout. A key focus of these materials should be on developing the clinician knowledge base, acceptance, and buy-in of the current guidelines.

Barrier: clinician learning curve for follow-up algorithm

Suggested solution(s)

Developing clear data visualizations of the new primary HPV testing follow-up algorithm to address clinicians’ and administrators’ questions (i.e., a pocket card or simple one-page flowsheet). Department administrators suggested nurses in particular respond to visual resources (e.g., visualizations of workflow).

Barrier: female nurses and physicians may have heightened resistance

Suggested solution(s)

Participants recommended remaining sensitive to the fact that many nurses and physicians are female and may have their own concerns or preconceptions about de-implementing the Pap test. Thus, it may be important to expend additional effort to gain buy-in from these clinicians.

Our findings can be applied to other health systems/settings considering implementation, particularly those within the USA or with a similar healthcare model (multiple payers, competition, concerns about patient loyalty, etc.). Studies from other countries that have implemented, or are currently implementing, primary HPV screening focus heavily on cost-effectiveness for national payers, cancer risks, and modeling downstream effects (e.g., potential increased colposcopy) and less on the concerns of clinician or patient buy-in and process change [11, 12, 51, 52]. Even in countries with a national health system organized around evidence-based care, there may be patient or clinician concerns about the change that our findings could help to address (e.g., patient concerns regarding unfamiliarity of primary HPV testing). Within the USA, there is the added layer of complexity that US guidelines currently recommend a suite of screening strategies with the new addition of primary HPV as an option (screening every 3 years with cytology alone, every 5 years with HPV testing alone, or every 5 years with HPV testing in combination with cytology (co-testing) in women 30–65 years, based on risk level) and European guidelines unequivocally recommend primary HPV screening [8]. Recommending multiple options for screening may compound the difficulties of implementation in the US context.

There are limitations to our findings. We used exploratory qualitative research methods to elicit participants’ perspectives. Consistent with this research approach, our study represents a small, but adequate sample size. While the findings may not be generalizable to all cervical cancer screening stakeholders within the KPSC region, our findings offer rich, context-specific insight into the experiences and perspectives of our clinical, administrator, and patient participants. We used standard and structured coding methods to extract themes from the raw data and employed independent coders to enhance analytic rigor.


Practice substitution to replace an entrenched clinical practice with a new alternative is complex, involving both de-implementation and implementation processes. Leveraging available facilitators is key to ease the process for clinical and administrative stakeholders—in this case, emphasizing the efficiency of going from two tests to one. Identifying and proactively addressing clinical and patient fears regarding perceptions of cost-cutting and poorer quality of care is critical and unique to this type of practice change. As with other efforts to facilitate practice change, multicomponent and multilevel strategies for engagement, education, and sustainment will be required.

Availability of data and materials

No datasets are available from this study owing to the consents given by participants, which limits data to the research team only.


  1. Curry SJ, Krist AH, Owens DK, Barry MJ, Caughey AB, Davidson KW, et al. Screening for cervical cancer: US Preventive Services Task Force recommendation statement. JAMA. 2018;320(7):674–86.

    Article  PubMed  Google Scholar 

  2. Updated cervical cancer screening guidlines: Practice Advisory. American College of Obstetricians and Gynecologists, April 2021; Accessed 14 Sept 2021.

  3. Ronco G, Dillner J, Elfstrom KM, Tunesi S, Snijders PJ, Arbyn M, et al. Efficacy of HPV-based screening for prevention of invasive cervical cancer: follow-up of four European randomised controlled trials. Lancet. 2014;383(9916):524–32.

    Article  PubMed  Google Scholar 

  4. Moyer VA. Screening for cervical cancer: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med. 2012;156(12):880–91, W312.

    Article  PubMed  Google Scholar 

  5. Massad LS, Einstein MH, Huh WK, Katki HA, Kinney WK, Schiffman M, et al. 2012 updated consensus guidelines for the management of abnormal cervical cancer screening tests and cancer precursors. Obstet Gynecol. 2013;121(4):829–46.

    Article  PubMed  Google Scholar 

  6. Saslow D, Solomon D, Lawson HW, Killackey M, Kulasingam SL, Cain J, et al. American Cancer Society, American Society for Colposcopy and Cervical Pathology, and American Society for Clinical Pathology screening guidelines for the prevention and early detection of cervical cancer. CA Cancer J Clin. 2012;62(3):147–72.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Wright TC, Stoler MH, Behrens CM, Sharma A, Zhang G, Wright TL. Primary cervical cancer screening with human papillomavirus: end of study results from the ATHENA study using HPV as the first-line screening test. Gynecol Oncol. 2015;136(2):189–97.

    Article  PubMed  Google Scholar 

  8. Maver PJ, Poljak M. Primary HPV-based cervical cancer screening in Europe: implementation status, challenges, and future plans. Clin Microbiol Infect. 2020;26(5):579–83.

    Article  CAS  PubMed  Google Scholar 

  9. Polman NJ, Snijders PJ, Kenter GG, Berkhof J, Meijer CJ. HPV-based cervical screening: rationale, expectations and future perspectives of the new Dutch screening programme. Prev Med. 2019;119:108–17.

    Article  CAS  PubMed  Google Scholar 

  10. Machalek DA, Roberts JM, Garland SM, Thurloe J, Richards A, Chambers I, et al. Routine cervical screening by primary HPV testing: early findings in the renewed National Cervical Screening Program. Med J Aust. 2019;211(3):113–9.

    Article  PubMed  Google Scholar 

  11. Jansen EEN, Naber SK, Aitken CA, de Koning HJ, van Ballegooijen M, de Kok IM. Cost-effectiveness of HPV-based cervical screening based on first year results in the Netherlands: a modelling study. BJOG. 2021;128(3):573–82.

    Article  PubMed  Google Scholar 

  12. Smith MA, Canfell K. Projected impact of HPV vaccination and primary HPV screening on cervical adenocarcinoma: example from Australia. Papillomavirus Research. 2017;3:134–41.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Hasson H, Nilsen P, Augustsson H, von Thiele Schwarz U. Empirical and conceptual investigation of de-implementation of low-value care from professional and health care system perspectives: a study protocol. Implement Sci. 2018;13(1):67.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Patey AM, Hurt CS, Grimshaw JM, Francis JJ. Changing behaviour ‘more or less’-do theories of behaviour inform strategies for implementation and de-implementation? A critical interpretive synthesis. Implement Sci. 2018;13(1):134.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Kerr EA, Kullgren JT, Saini SD. Choosing wisely: how to fulfill the promise in the next 5 years. Health Aff (Millwood). 2017;36(11):2012–8.

    Article  PubMed  Google Scholar 

  16. McKay VR, Morshed AB, Brownson RC, Proctor EK, Prusaczyk B. Letting go: conceptualizing intervention de-implementation in public health and social service settings. Am J Community Psychol. 2018;62(1-2):189–202.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Augustsson H, Ingvarsson S, Nilsen P, von Thiele SU, Muli I, Dervish J, et al. Determinants for the use and de-implementation of low-value care in health care: a scoping review. Implement Sci Commun. 2021;2(1):1–7.

    Article  Google Scholar 

  18. Norton WE, Chambers DA, Kramer BS. Conceptualizing de-implementation in cancer care delivery. J Clin Oncol. 2019;37(2):93–6.

    Article  PubMed  Google Scholar 

  19. Blayney DW, Simon MK, Podtschaske B, Ramsey S, Shyu M, Lindquist C, et al. Critical lessons from high-value oncology practices. JAMA Oncol. 2018;4(2):164–71.

    Article  PubMed  Google Scholar 

  20. Rocque G, Blayney DW, Jahanzeb M, Knape A, Markham MJ, Pham T, et al. Choosing wisely in oncology: are we ready for value-based care? J Oncol Pract. 2017;13(11):e935–e43.

    Article  PubMed  Google Scholar 

  21. Hahn EE, Munoz-Plaza C, Wang J, Garcia Delgadillo J, Schottinger JE, Mittman BS, et al. Anxiety, culture, and expectations: oncologist-perceived factors associated with use of nonrecommended serum tumor marker tests for surveillance of early-stage breast cancer. J Oncol Pract. 2017;13(1):e77–90.

    Article  PubMed  Google Scholar 

  22. Skolarus TA, Hawley ST, Wittmann DA, Forman J, Metreger T, Sparks JB, et al. De-implementation of low value castration for men with prostate cancer: protocol for a theory-based, mixed methods approach to minimizing low value androgen deprivation therapy (DeADT). Implement Sci. 2018;13(1):144.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Sharp AL, Klau MH, Keschner D, Macy E, Tang T, Shen E, et al. Low-value care for acute sinusitis encounters: who’s choosing wisely? Am J Manag Care. 2015;21(7):479–85.

    PubMed  Google Scholar 

  24. Born K, Kool T, Levinson W. Reducing overuse in healthcare: advancing Choosing Wisely. BMJ. 2019;367:l6317.

    Article  PubMed  Google Scholar 

  25. Wang T, Baskin AS, Dossett LA. Deimplementation of the choosing wisely recommendations for low-value breast cancer surgery: a systematic review. JAMA Surg. 2020;155(8):759–70.

    Article  PubMed  Google Scholar 

  26. Rietbergen T, Spoon D, Brunsveld-Reinders AH, Schoones JW, Huis A, Heinen M, et al. Effects of de-implementation strategies aimed at reducing low-value nursing procedures: a systematic review and meta-analysis. Implement Sci. 2020;15(1):38.

    Article  PubMed  PubMed Central  Google Scholar 

  27. van Bodegom-Vos L, Davidoff F. Marang-van de Mheen PJ. Implementation and de-implementation: two sides of the same coin? BMJ Qual Saf. 2017;26(6):495–501.

    Article  PubMed  Google Scholar 

  28. Wang V, Maciejewski ML, Helfrich CD, Weiner BJ. Working smarter not harder: coupling implementation to de-implementation. Healthc (Amst). 2018;6(2):104–7.

    Article  PubMed  Google Scholar 

  29. Helfrich CD, Rose AJ, Hartmann CW, van Bodegom-Vos L, Graham ID, Wood SJ, et al. How the dual process model of human cognition can inform efforts to de-implement ineffective and harmful clinical practices: a preliminary model of unlearning and substitution. J Eval Clin Pract. 2018;24(1):198–205.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217–26.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Fisher ES, Shortell SM, Savitz LA. Implementation science: a potential catalyst for delivery system reform. Jama. 2016;315(4):339–40.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Gupta DM, Boland RJ Jr, Aron DC. The physician’s experience of changing clinical practice: a struggle to unlearn. Implement Sci. 2017;12(1):28.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Koebnick C, Langer-Gould AM, Gould MK, Chao CR, Iyer RL, Smith N, et al. Sociodemographic characteristics of members of a large, integrated health care system: comparison with US Census Bureau data. Permanente J. 2012;16(3):37–41.

    Article  Google Scholar 

  34. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the Consolidated Framework for Implementation Research. Implement Sci. 2016;11:72.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Hahn EE, Munoz-Plaza CE, Schottinger JE, Brasfield FM, Gould MK, Parry C. Developing innovative models of care for cancer survivors: use of implementation science to guide evaluation of appropriateness and feasibility. Support Care Cancer. 2019;27(5):1737–45.

  37. Munoz-Plaza CE, Parry C, Hahn EE, Tang T, Nguyen HQ, Gould MK, et al. Integrating qualitative research methods into care improvement efforts within a learning health system: addressing antibiotic overuse. Health Res Policy Syst. 2016;14(1):63.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Francis JJ, Johnston M, Robertson C, Glidewell L, Entwistle V, Eccles MP, et al. What is an adequate sample size? Operationalising data saturation for theory-based interview studies. Psychol Health. 2010;25(10):1229–45.

    Article  PubMed  Google Scholar 

  39. Guest G, Bunce A, Johnson LJFM. How many interviews are enough? vol. 18; 2006. p. 59–82.

    Google Scholar 

  40. Hennink MM, Kaiser BN, Weber MB. What influences saturation? Estimating sample sizes in focus group research. Qual Health Res. 2019;29(10):1483–96.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Saunders B, Sim J, Kingstone T, Baker S, Waterfield J, Bartlam B, et al. Saturation in qualitative research: exploring its conceptualization and operationalization. Qual Quan. 2018;52(4):1893–907.

    Article  Google Scholar 

  42. MacQueen KM, McLellan E, Kay K, Milstein B. Codebook development for team-based qualitative analysis. CAM J. 2016;10(2):31–6.

    Article  Google Scholar 

  43. Fulton BD. Health care market concentration trends in the United States: evidence and policy responses. Health Affairs. 2017;36(9):1530–8.

    Article  PubMed  Google Scholar 

  44. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10(1):21.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Mazza D, Bairstow P, Buchan H, Chakraborty SP, Van Hecke O, Grech C, et al. Refining a taxonomy for guideline implementation: results of an exercise in abstract classification. Implement Sci. 2013;8(1):32.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Leeman J, Baernholdt M, Sandelowski M. Developing a theory-based taxonomy of methods for implementing change in practice. J Adv Nursing. 2007;58(2):191–200.

    Article  Google Scholar 

  47. Walter I, Nutley S, Davies H. Developing a taxonomy of interventions used to increase the impact of research; 2003.

    Google Scholar 

  48. Prusaczyk B, Swindle T, Curran G. Defining and conceptualizing outcomes for de-implementation: key distinctions from implementation outcomes. Implement Sci Commun. 2020;1(1):43.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Norton WE, Chambers DA. Unpacking the complexities of de-implementing inappropriate health interventions. Implement Sci. 2020;15(1):2.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Patey AM, Grimshaw JM, Francis JJ. Changing behaviour, ‘more or less’: do implementation and de-implementation interventions include different behaviour change techniques? Implement Sci. 2021;16(1):1–7.

    Article  Google Scholar 

  51. Yoshida Y, Iwanari O, Oishi T, Kasai T, Hamada M, Fujita H, et al. Implementation of primary HPV testing in Japan. Mol Clin Oncol. 2020;13(4):22.

    PubMed  PubMed Central  Google Scholar 

  52. Mensah K, Kabore C, Zeba S, Bouchon M, Duchesne V, Pourette D, et al. Implementation of HPV-based screening in Burkina Faso: lessons learned from the PARACAO hybrid-effectiveness study. BMC Women’s Health. 2021;21(1):1–6.

    Article  Google Scholar 

Download references


We would like to thank our patient stakeholder group for their input on our interview guide, and our clinical stakeholder group for their input and guidance on our barriers and facilitators and interview guide. We would also like to thank our interview participants for giving of their time for our study.


Research reported in this manuscript was funded through a Patient-Centered Outcomes Research Institute® (PCORI®) Award (CDR-2018C1-10987).

Author information

Authors and Affiliations



CRC, DT, BSM, EEH, MKG, and CMP generated the idea for the study and designed the study. CMP and EEH led the drafting of the manuscript. CH and NTC prepared approval applications, provided project management, contributed to the interview guide, and coordinated recruitment. DT, LHA, QNM, MH, PW, KST, and RNE contributed clinical expertise to the interview guide. CMP and DEA oversaw the development of the interview guide and conducted interviews. CMP, EEH, and CH conducted the analyses. All authors contributed to the drafting of the manuscript and read and approved the final manuscript.

Corresponding author

Correspondence to Erin E. Hahn.

Ethics declarations

Ethics approval and consent to participate

This study has been approved by the Kaiser Permanente Southern California Institutional Review Board (IRB) #12015. We obtained verbal consent from all participants.

Consent for publication

All participants gave permission for their comments to be published in an anonymized form.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

COREQ (COnsolidated criteria for REporting Qualitative research) Checklist.

Additional file 2.

Implementing Primary HPV Testing Patient Interview Guide.

Additional file 3.

Implementing Primary HPV Testing Physicians, Nurses, and Administrators Interview Guide.

Additional file 4.

List of Potential Barriers – Primary HPV Testing for Routine Cervical Cancer Screening.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hahn, E.E., Munoz-Plaza, C., Altman, D.E. et al. De-implementation and substitution of clinical care processes: stakeholder perspectives on the transition to primary human papillomavirus (HPV) testing for cervical cancer screening. Implement Sci Commun 2, 108 (2021).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: