- Open Access
Evaluation of multi-level barriers and facilitators in a large diabetic retinopathy screening program in federally qualified health centers: a qualitative study
Implementation Science Communications volume 2, Article number: 54 (2021)
Recommended annual diabetic retinopathy (DR) screening for people with diabetes has low rates in the USA, especially in underserved populations. Telemedicine DR screening (TDRS) in primary care clinics could expand access and increase adherence. Despite this potential, studies have observed high variability in TDRS rates among clinics and over time, highlighting the need for implementation supports. Previous studies of determinants of TDRS focus on patients’ perspectives, with few studies targeting upstream multi-level barriers and facilitators. Addressing this gap, this qualitative study aimed to identify and evaluate multi-level perceived determinants of TDRS in Federally Qualified Health Centers (FQHCs), to inform the development of targeted implementation strategies.
We developed a theory-based semi-structured interview tool based on the Consolidated Framework for Implementation Research (CFIR). We conducted 22 key informant interviews with professionals involved in TDRS (administrators, clinicians, staff). The interviews were audio-recorded and transcribed verbatim. Reported barriers and facilitators were organized into emergent themes and classified according to CFIR constructs. Constructs influencing TDRS implementation were rated for each study site and compared across sites by the investigators.
Professionals identified 21 main barriers and facilitators under twelve constructs of the five CFIR domains. Several identified themes were novel, whereas others corroborated previous findings in the literature (e.g., lack of time and human resources, presence of a champion). Of the 21 identified themes, 13 were classified under the CFIR’s Inner Setting domain, specifically under the constructs Compatibility and Available Resources. Themes under the Outer Setting domain (constructs External Incentives and Cost) were primarily perceived by administrators, whereas themes in other domains were perceived across all professional categories. Two Inner Setting (Leadership Engagement, Goals and Feedback) and two Process (Champion, Engaging) constructs were found to strongly distinguish sites with high versus low TDRS performance.
This study classified barriers and facilitators to TDRS as perceived by administrators, clinicians, and staff in FQHCs, then identified CFIR constructs that distinguished high- and low-performance clinics. Implementation strategies such as academic detailing and collection and communication of program data and successes to leadership; engaging of stakeholders through involvement in implementation planning; and appointment of intervention champions may therefore improve TDRS implementation and sustainment in resource-constrained settings.
Diabetic retinopathy (DR) is the leading cause of blindness in working age adults in the USA , and its timely detection and treatment reduce the risk of severe vision loss [2,3,4]. Success with early intervention is the basis for the DR screening guidelines of the American Academy of Ophthalmology, the American Diabetes Association, and the International Council of Ophthalmology, which recommend an annual (or in some cases biennial) eye exam or retinal photograph interpreted by an experienced reader [5,6,7]. Despite guidelines, as few as 18–33% of people in US communities with inequitable access to eye care (such as urban poor and rural communities) receive adequate diabetic retinopathy screening [8, 9]. Even among insured populations, as few as 15% of people with diabetes with no previously diagnosed DR receive adequate screening .
Telemedicine DR screening (TDRS) has been used for decades in national healthcare systems, such as the National Health System in the UK , as well as the Veterans Health Administration system in the USA [12, 13]. When screening achieves high rates of uptake and adherence to follow-up and treatment, diabetic eye disease can be dislodged as the leading cause of certifiable blindness among working age adults .
In the US, TDRS is increasingly being deployed to reach people with diabetes who otherwise may not adhere to recommended DR screenings . With this technology, a non-mydriatic camera installed in a primary care clinic can acquire retina images during routine care visits of people with diabetes eligible for annual screening. Images are transmitted for interpretation by a remote reader, and a report is issued and sent to the requesting clinician with a recommendation for follow-up, tailored to the pathology found .
This approach eliminates several patient-level barriers associated with accessing in-person eye exams conducted by a specialist. Patient inconvenience, accessibility of the screening clinic (issues with transport and distance), time, and difficulty scheduling appointments have been identified as major barriers to DR screening [16,17,18,19,20]. For patients who attend primary care appointments, some barriers to screening may persist, such as financial concerns, competing health problems, and lack of symptoms , but many are eliminated when TDRS is offered during the appointment .
TDRS can have the most impact in low-resource populations, where factors such as access and patient education negatively impact DR screening rates [17, 22,23,24,25]. While the nature of TDRS alleviates patient-level barriers to screening, its integration into primary care clinical workflows is inconsistent, often suboptimal , with undefined best practices for implementation. Indeed, a significant proportion of people with diabetes treated in clinics equipped with TDRS technology remain unscreened [27, 28], suggesting that undocumented barriers may exist at a different level, for example, that of professionals involved.
To improve TDRS implementation in primary care clinics, more insight is needed regarding specific barriers and facilitators in this setting. Previous TDRS research has mainly focused on patients, with scarce data addressing barriers at the level of healthcare professionals (clinicians, nurses, staff) , organizations, or systems. Multi-level barriers and facilitators are particularly important in low-resource clinics where the decision to administer the exam is frequently made by staff rather than clinicians, and where priorities for resource allocation and patient care have to be carefully weighed.
The Consolidated Framework for Implementation Research (CFIR)  guided our conceptualization of multi-level factors influencing implementation of TDRS. The CFIR is a comprehensive, meta-theoretical framework of 39 constructs organized across five major domains theorized to influence implementation . Importantly, these domains are multi-level and allow investigation of factors influencing implementation above the more commonly studied patient level. In this study, we addressed the five CFIR domains, as they relate to characteristics of clinicians, staff, and administrators involved in TDRS (Individuals Involved); the TDRS intervention (Intervention Characteristics); FQHCs (Inner Setting); the broader healthcare system (Outer Setting); and strategies for roll-out and operational integration (Process).
Implementation science approaches have rarely been applied in the field of ophthalmology [32,33,34,35], but offer powerful frameworks and methods to increase the public health impact of effective, yet under-utilized, eye care interventions. Thus, the goals of this study were to (1) use the CFIR to identify clinician-, staff-, organizational-, and systems-level barriers and facilitators to TDRS in low-resource primary care settings, and (2) identify influential CFIR constructs that explain implementation variability in this setting, operationalized by rates of TDRS delivery. We specifically targeted Federally Qualified Health Centers (FQHCs) serving urban poor and rural populations, and we conducted our data collection and analysis guided by the CFIR. The systematic identification of contextual factors associated with TDRS implementation is necessary to inform the next step in our program of research: the selection of tailored implementation strategies aimed at increasing the adoption and sustainment of TDRS in primary care clinics.
The Standards for Reporting Qualitative Research (SRQR) checklist was used to guide reporting of methods and findings (Additional file 1). For complete methods, please see Additional file 2.
Eligible clinics were FQHCs in an existing TDRS network described elsewhere , ensuring that all had some experience with the service. In selecting the sites for our study, we used maximum variation purposive sampling to obtain a diverse mix of clinic characteristics. We included a total of six clinics serving either rural (3 clinics) or urban (3 clinics) low-income communities. Additionally, we divided our clinical practices into three strata related to DR screening rates (high, medium, or low), and included 2 clinics from each stratus.
We applied maximum variation purposive sampling for the selection of individual participants as well, selecting from three types of professionals (i.e., clinicians, staff, and administrators) and including individuals with varying years in practice. The selected individuals received an emailed invitation to participate in the study. If a selected individual declined to participate, we invited another professional from the same clinic.
A semi-structured interview guide was developed to allow participants to talk freely and volunteer rich information. A draft version was developed and pilot-tested. Modifications to the interview guide were made to improve question clarity and add newly identified lines of inquiry. The final semi-structured interview guide targeted constructs within all five CFIR domains (Additional file 3).
Sociodemographic characteristics of participants and clinics were collected using self-report via standard items, and included age, gender, race, ethnicity, professional role, and years in profession. Measured characteristics of clinics were provided by clinics’ administrators and included urban versus rural setting, and DR screening rate.
Face-to-face key informant interviews were conducted between August 2018 and March 2019. All interviews were audio-recorded and digital recordings were transcribed verbatim. The number of interviews completed was determined by data saturation: iterative coding was conducted as interviews progressed, with no new themes identified in the final two transcripts .
Data coding and analysis
The investigative team developed a preliminary codebook for conventional content analysis through a process of discussion and refinement . Briefly, trained researchers independently coded a sample of 3 transcripts using version I of the codebook and ATLAS.ti 8.4 software. The results of each coding were used to refine the codebook. Using version II of the codebook, the raters re-coded the first set of transcripts. When consensus was reached on a final codebook version and on coding of the sample transcripts, the coding process continued for the rest of the transcripts and a grid of emergent themes was developed. Two participants were invited for a member-checking process , following which the grid of themes was reviewed. Themes were then categorized into a CFIR-based matrix of relevant constructs and domains. For a complete method, see Additional file 2.
The identification of influential constructs followed methods described by Damschroder et al. . Briefly, implementation effectiveness was characterized as high (2 sites), medium (2 sites), or low (2 sites), and a memo organized by CFIR construct was created for each site. Constructs were rated (as − 2, − 1, 0, + 1, + 2) based on the strength and valence of influence as perceived by key informants) through a deliberated consensus process for each memo (site). Ratings were compared across the two high and the two low implementation facilities to identify patterns in constructs distinguishing high and low implementation effectiveness. For a complete method, see Additional file 2.
Of the 24 healthcare professionals contacted, 22 (92%) agreed to participate and completed the interview. The two who declined were primary care physicians (one male, one female).
Table 1 summarizes the characteristics of the professionals interviewed and clinic characteristics. The participating professionals varied widely in age and years of experience. Most were non-Hispanic White (n = 16/22), female (n = 21/22), and aged 40 or older (n = 15/22). In-profession experience ranged from 4 to 42 years, and the average number of people with diabetes seen weekly ranged from 6 to 75. Most participants worked in urban clinics (n = 15/22) and were distributed among clinics with low (n = 10/22), medium (n = 7/22), and high (n = 5/22) TDRS rates. Time of clinic use of TDRS services ranged between 1 and 4.5 years.
Qualitative themes by CFIR domain and professional role
Twenty-one themes emerged from interviews, which corresponded to twelve CFIR constructs and five domains (Fig. 1). Most themes described by participants were grouped under the domain Inner Setting (13 of 21 themes). Results are presented below, organized by CFIR domains and constructs, and by professional role (administrators, clinicians, staff) (Fig. 1).
The first of two themes in this domain concerned perceptions of patients’ attitudes towards TDRS, consistent with the CFIR construct Knowledge and Beliefs (Fig. 1). Participants believed that the majority of patients were motivated by the convenience and ease of completing the diabetic eye exam in the primary care clinic:
Low-income patients especially, like a lot of the migrant patients, they just, they feel comfortable here. […] I’ve found that it’s been very nice to like, at least get a screening test done and have that available here. (Participant 9, clinician, urban clinic, low DR screening rate)
The second theme was participants’ comfort with the exam. Degree of comfort or confidence performing TDRS was perceived as either a barrier (for low degree) or facilitator (for high degree), and this theme aligned with the CFIR construct Self-Efficacy (Fig. 1). One staff supervisor noted how self-efficacy could influence screening, as staff with less TDRS experience and lower degree of confidence would only offer the exam to eligible patients if instructed by the clinician (frequently referred to as provider in the USA):
Many of my more experienced CMAs they feel more comfortable to go ahead and do it themselves before provider will say it. And for some of those who are newer, they will do it only when the provider will remind them. (Participant 4, staff supervisor, urban clinic, low DR screening rate)
Two themes were associated with this domain: exam gradeability and exam length. These themes both reflect the CFIR construct Complexity of the Intervention (Fig. 1).
Many participants described gradeability of TDRS (whether the exam result is of sufficient quality to be interpreted) as a challenge. While the device provides user guidance and indicates whether the image obtained is adequate, acquiring a gradable picture required some user experience:
As simple as the machine is and as automated as it is, there is still room for error. […] If you do it wrong you can get a bad picture, so that’s probably the biggest challenge. (Participant 2, administrator, urban clinic, medium DR screening rate)
Similarly, the length of the exam was also perceived as a barrier by some clinicians and staff:
Out of all the diabetic screenings, that one is the more time-consuming one. (Participant 19, staff, urban clinic, low DR screening rate)
Several participants described exam length as a frequent reason for not offering TDRS to patients, especially in clinics in which TDRS is performed by the same staff who prepare patients for their clinical encounters. This perception was not universal, however. Other participants described needing only a few minutes to complete the exam, stating that the convenience and necessity of TDRS outweighed the potential disruption to clinic flow.
Most identified themes fell under this domain, and they clustered around 6 CFIR constructs: (1) Relative Priority, (2) Available Resources, (3) Compatibility, (4) Leadership Engagement, (5) Goals and Feedback, and (6) Access to Knowledge and Information (Fig. 1).
Many clinicians and staff described the priority of TDRS as low relative to other exams necessary for people with diabetes, particularly for those with multiple pathologies. This low relative priority was clear in several participants’ descriptions of screening decisions:
They come in with a million problems […], and most of the time you just have to prioritize which are more important, and they always have a lot of acute problems that need attention and so things like screening sometimes just get dropped on the bottom. (Participant 8, clinician, urban clinic, low DR screening rate)
Several participants mentioned that changes in other clinic workflows and systems that coincided with TDRS implementation made it more challenging to adopt the exam, suggesting that low relative priority was given to TDRS implementation:
We’ve been using these EMR forms for a little over six months, and we’ve changed something a couple weeks ago. And people are now like, ‘another change…’, you know, so they’ve had to adapt to that. (Participant 1, administrator, urban clinic, low DR screening rate)
Professionals described lack of resources and time as a barrier to TDRS. In most participating clinics, TDRS was performed by medical assistants (healthcare workers trained for the role of clinicians in tasks such as taking medical histories, recording vital signs, and other administrative and clinical tasks). This was in addition to their other responsibilities (such as those detailed above), and this lack of dedicated personnel to perform TDRS was perceived as an impediment:
It comes down to the time piece. If it just feels overwhelming, if I’m overwhelmed and they’re overwhelmed […] and then there’s this diabetic eye exam, that may fall through the cracks. (Participant 9, clinician, urban clinic, low DR screening rate)
The existence of dedicated staff for TDRS was considered a potential facilitator by some professionals:
In my wish list, the nurse I would have just for diabetic patients, for education, and this nurse will be […] bringing patients and doing this [TDRS exam] herself and making sure of quality. (Participant 4, staff supervisor, urban clinic, low DR screening rate)
Availability of physical space was another key theme. This TDRS system required a dark room and space for the desktop camera, and participants considered existence of adequate space as a facilitator. Similarly, some clinics allocated resources to improving TDRS workflow after an initial trial period (e.g., the camera was moved to a more convenient location; the exam room was darkened with curtains). Consistently, difficulties finding appropriate space for the camera posed challenges for implementation efforts.
Resource scarcity was also described as influencing communication between clinics and external clinicians. Specifically, participants attributed issues with retrieving reports of eye exams performed elsewhere to a lack of staff. Difficulties documenting whether screening had been performed elsewhere acted as a barrier to TDRS, since most professionals would not order it if patients mentioned having an eye exam in the past 12 months (even if this was not documented):
The provider has said I’m not going to send them for an eye screen if they’re telling me they’ve already had one, but then maybe their health reminder is not satisfied because we can’t get the records. (Participant 7, staff, urban clinic, medium DR screening rate)
Some participants also reported difficulties accessing an eye care specialist for patients with referable pathology found in TDRS, explaining that this translated into frustration with providing screening:
Once they have an abnormal scan […] I’m going to send them to an eye doctor anyway. So the issue I have is getting them to go back to the eye doctor […] because that’s another appointment and it's outside. (Participant 22, clinician, urban clinic, medium DR screening rate)
Compatibility of TDRS with clinic systems and workflows was identified as both a facilitator and a barrier. The reminder for TDRS was seen as a facilitator when it was considered reliable, which happened mostly when the reminder was activated by a staff who “scrubbed” charts (i.e., manually reviewed the chart prior to the appointment):
We do have scrub sheets here that we are able to look at, that they let us know if certain things need to be done at the time of their visit, so if “eye exam” is marked on there, we can try to go ahead and perform that. (Participant 16, staff, rural clinic, high DR screening rate)
However, one administrator mentioned that electronic medical records (EMR) reminders were not trusted, as they were automated and lacked integration, which left clinicians without a reliable reminder for TDRS.
All clinics in the study used the standing order method of request for TDRS. With this method, the exam order was conditioned upon the occurrence of certain criteria—a person with diabetes without a diabetic eye exam in the past 12 months—and patients who met these criteria could receive the exam without the need for a physician order. One administrator noted that, regardless of a standing order, when TDRS was specifically requested by a clinician (not left to the discretion of the staff to perform), exam rates increased. Thus, request by clinician was seen as a facilitator, but one that was not always compatible with usual clinic procedures and flow.
In a parallel theme of compatibility with existing workflows, lack of integration of exam reports into the existing EMR system was an identified barrier.
For something like a retinal image that’s read outside and we get a scanned document back in, [the EMR] doesn’t automatically populate those fields. And so we’ve done a lot of work to try to develop an abstracting system, so that when we get those [TDRS] reports back, not only are we scanning those in for the provider to review and sign off, but we’re also entering the completion date into a reportable field. (Participant 11, administrator, rural clinics, medium and high DR screening rates)
Leadership engagement and goals and feedback
These CFIR constructs tended to appear jointly in interviews. Most participants perceived engagement and feedback from leadership as facilitators, explaining that clinic directors and administrators had high engagement in this exam:
If you are falling way low you get, you know, emails from the medical director that you need to get your numbers of your diabetics, and your diabetic retinopathy scans done. (Participant 10, clinician, rural clinic, medium DR screening rate)
When clinicians were described as leaders, their engagement was identified as either a barrier or a facilitator, depending on whether they reinforced and supported staff in initiating TDRS:
The focus is more on controlling the hemoglobin A1C, so I think that’s probably is what people look at, at the physician level, and I think what the physician says, kind of flows down to the support staff. (Participant 1, administrator, urban clinic, low DR screening rate)
It all ends up being in our, like the providers’ hands, basically more or less. The staff know that they are supposed to do [the TDRS exam], and periodically they are reinforced to. (Participant 8, clinician, urban clinic, low DR screening rate)
Several participants explained that sharing data regarding TDRS rates would be a valuable way of providing feedback and increasing adherence to the exam:
Data is helpful when you go back and review […]. I can send this out, the graph out showing how many we’re doing a month. I can report that out to our providers and show them where we are (Participant 6, staff, urban clinic, medium DR screening rate).
Access to knowledge and information
While TDRS is considered easy to perform by even non-eye care professionals, many staff felt like their training had been insufficient:
We weren’t really sure what were the appropriate images, what was acceptable, what wasn’t. Again, it seemed like we had to kind of learn more of that on our own, so I do think there could have been a little bit more training. (Participant 6, staff, urban clinic, medium DR screening rate)
Participants also believed that the training plan—one initial stand-alone session without refreshers—was inadequate in clinics with high personnel turnover. This left new staff either untrained or trained by a colleague, which was felt to be less adequate than the training provided initially by the expert trainer.
Within this domain, the primary identified theme was the lack of a financial driver for TDRS, despite its relatively low cost, consistent with the CFIR construct External Policies and Incentives (Fig. 1). Administrators described complexities in obtaining reimbursement for TDRS, which typically resulted in clinics not billing for performed screenings. Additionally, while Medicare and Medicaid include DR exam as a quality measure, this potential facilitator was described as having minimal influence, due to low economic incentives for attaining these metrics.
There’s not a pressing financial driver for these things.[…] There’s a lot of talk about in the future getting paid more for quality. Right now the way the finances are I’m not aware that there’s a driving thing (Participant 2, administrator, urban clinic, medium DR screening rate).
Themes in this domain were consistent with the CFIR constructs Engaging and Champion (Fig. 1).
This CFIR construct refers to efforts to attract and involve individuals in the use of an intervention through activities such as marketing, training, or education . A lack of engagement was noted by some clinicians and staff, describing uncertainty with the sensitivity of the technology. Participants suggested that access to education about the importance of DR screenings could work as a facilitating tool:
They need to feel empowered to make that decision and why they’re doing it. It’s not just another checked box on their list, that “They’re diabetic, I have to do this”. […]. If we had more education for them, for them the support staff, it may become, it may help us bump that priority side. (Participant 3, staff supervisor, urban clinic, low DR screening rate)
While most clinics in the study did not have a recognized champion, participants agreed that such an agent would likely promote use of TDRS. Within the few clinics that did have a champion, all participants identified that individual as an influential facilitator:
She’s the one that makes sure that they’re done to the best of their abilities and that people are doing [the TDRS exams] right. […] She made it her mission and went above and beyond for it. (Participant 22, clinician, urban clinic, medium DR screening rate)
Perceptions of barriers and facilitators by professional role
We found that the majority of themes were identified across all professional roles (Fig. 1). When analyzed by CFIR domain, themes falling into the Outer Setting (construct External Policies and Incentives) were mostly perceived by administrators. Conversely, themes aligned with the domain Intervention Characteristics (construct Complexity) were emphasized by clinicians and staff. Staff were also more likely to refer to issues with comfort performing the exam (Individuals Involved domain, construct Self-efficacy), as well as to difficulties accessing DRS reports for patients who were screened by an eye care clinician (Inner Setting domain, construct Available resources).
Identification of constructs distinguishing sites with low versus high TDRS implementation effectiveness
We assessed the 12 CFIR constructs that were consistent with the identified themes and rated each as having positive or negative influence on TDRS performance (Table 2). We found that four constructs were perceived as strongly distinguishing low from high implementation effectiveness, defined by rates of TDRS delivery. Two were related to Inner Setting (Leadership Engagement, and Goals and Feedback), and two were related to Process (Engaging and Champion). Two additional constructs weakly distinguished sites (Relative Priority and Available Resources), and four were non-distinguishing (Complexity, Access to Knowledge and Information, Compatibility, and External Policies and Incentives). We considered the constructs Knowledge and Beliefs and Self-Efficacy (Individual domain) as not applicable, because the focus of our ratings was not on individual-level behavior change but rather on clinic-level characteristics. Additional file 4 provides quotes and memo details on how each construct manifested in the study sites.
Despite the potential for TDRS to increase DR screening rates in primary care settings, its adoption is often suboptimal. This qualitative study (1) identified perspectives of staff, clinicians, and administrators in FQHCs regarding barriers and facilitators to TDRS, (2) determined differences in perceptions of barriers and facilitators by professional category, and (3) prioritized CFIR constructs based on their perceived influence on implementation effectiveness, with the goal of informing implementation and sustainment strategies towards improved patient outcomes.
Our findings that constructs perceived to distinguish high versus low implementation sites cluster in the Inner Setting and Process domains of CFIR are consistent with implementation research on other evidence-based practices and interventions [39,40,41,42] and indicate influential targets for implementation strategies.
Common barriers to TDRS implementation
Resource constrains were observed to affect exam performance, accessing prior results, making referrals, and tracking screenings over time. This finding is unsurprising given previous evidence that resource scarcity hinders sustained program implementation of evidence-based interventions in general in low-resource environments . Specific to TDRS, challenges in obtaining timely referrals and treatment for patients with pathology found through the screening have previously been noted as barriers , consistent with our findings.
Poor integration of TDRS with existing EMR systems compounds the perceived lack of resources, as it requires extra time and effort in an already strained environment. Technological challenges perceived as increasing workload are a common barrier to telemedicine uptake in general . Software solutions that integrate TDRS devices and outputs into EMRs are necessary but may be costly. Identification of less expensive strategies to target this barrier, such as additional training or interventions that support self-efficacy towards new technology, may be needed.
Low relative priority is a common cause of poor implementation of a wide range of innovations in primary care , and this was the case for TDRS as well. Combined with the stresses of inadequate resources dedicated to TDRS, low relative priority and low motivation reported by clinicians and staff may be important targets for intervention.
In addition to the Inner Setting themes discussed above, professionals identified exam complexity as a barrier to consistent use of TDRS. In most clinics, workflow was not formally modified to accommodate this new test, which likely exacerbated these perceptions. Instead, workaround strategies (informal temporary practices for handling exceptions to normal workflow)  were frequently used, and staff were assigned TDRS as an additional task to squeeze into their day. While workarounds are commonly used in medical settings , and specifically in TDRS services , they can increase medical errors  and create further load in already resource-constrained clinics . Therefore, while sometimes necessary, workarounds should be minimal, temporary, and replaced with formal workflow adjustments  as soon as possible.
At the level of individuals, several staff and clinicians expressed low self-efficacy in performing TDRS, linked with perceptions that their training had been inadequate. Training of personnel is a basic and necessary implementation strategy , but alone it is generally insufficient . Consideration of enhanced training modalities and supports is warranted to target this barrier and may include strategies suggested by key informants, such as more training at inception of TDRS, continued training sessions going forward, and easily accessible videos or manuals for troubleshooting.
Distinguishing constructs as primary targets for implementation strategies
The process of selecting targets for generalizable implementation strategies can be fraught with difficulty, due to variation in barriers and to challenges matching strategies to targets for change. One pragmatic approach is to prioritize contextual factors (or constructs) that distinguish between high- and low-implementing sites in observational analyses. Targeting such constructs may lead to more effective implementation and sustainment of TDRS.
In this study, we consistently found that in the Inner Setting domain, the constructs Leadership Engagement and Goals and Feedback strongly distinguished high and low implementation sites. Indeed, engaged and consistent leadership in primary care can have a positive impact on adoption of specific practices, whereas low engagement by leaders can present as a barrier . Stakeholder feedback highlighted the importance of engaged leaders communicating with clinicians and staff about TDRS and providing frequent and effective reinforcement and feedback based on program data.
Also in the Inner Setting domain, the constructs Relative Priority and Available Resources, which have also been shown to influence implementation of evidence-based practices across settings [39, 52], similarly distinguished between high- and low-performing sites, though less strongly. While resources were constrained at all sites, clinics categorized as high- and medium-performing sites with respect to TDRS rates were mostly able to overcome resource scarcity by reallocating personnel and space, adapting the EMR system, and redesigning workflows. These adjustments seemed to be motivated by higher priority of TDRS in these sites, where the exam was seen as equally important to other diabetic screenings. Further, by anticipating the inevitable resource burdens  of a new exam, these measures may have contributed to reduction in the perceived complexity of the intervention , with more seamless adoption and sustainment.
Our findings regarding these four Inner Setting constructs suggest that implementation of TDRS in low-resource settings may benefit from interventions targeting leadership engagement (such as collection and communication of program data and successes; academic detailing) and appropriate planning of required resources (reallocation of space, funds, and personnel time). Targeting such factors may lead to a heightened perception of relative priority by stakeholders and a stronger implementation climate .
In the Process domain, the constructs Engaging and Champion strongly distinguished implementation effectiveness. The value of upfront planning for implementation of new technology or telemedicine services, especially in resource-constrained settings, cannot be overstated . Sites with more thorough implementation planning, which included engaging of stakeholders (i.e., staff and clinicians who would be directly involved in TDRS), were more likely to have higher implementation effectiveness, consistent with studies in other fields . Further, sites with TDRS champions had more active promotion of TDRS, quicker resource mobilization, and more efficient feedback about TDRS to health professionals. These Process factors are critical for implementing change in primary care , and affirm the importance of implementation strategies for TDRS in FQHCs that engage stakeholders (e.g., education, involving stakeholders in planning, tailoring TDRS to prospective barriers, and designating a project champion).
While our analysis identified perceived factors that distinguished between high and low implementation effectiveness, our study design does not permit us to determine which of these are most important for TDRS implementation, nor how they are associated with characteristics of each profession. As with any qualitative study, social desirability bias is a possibility; however, participants seemed frank and genuine in relaying their experiences with TDRS in their clinics. Our sampling strategy precludes generalizability of results beyond the participants in this study; however, our purposeful selection of clinics and key informants ensured a range of experiences and perceptions among study participants. In one of six sites, we were not able to attribute ratings to the constructs Engaging and Champion. This may decrease the strength of the evidence that these constructs strongly distinguish sites. Nonetheless, the pattern of ratings across low, medium, and high sites for these constructs is striking and may suggest a dose-response relative to site performance.
We plan to address the inherent limitations of qualitative research in the next phase of our research, in which quantitative survey methods will be used to investigate the prevalence and relative importance of themes identified in this study among a representative sample of health professionals and practice sites in our network.
Despite these limitations, this study offers insight into perceived challenges for adoption of TDRS in primary care clinics serving rural and urban poor populations in the USA, where the telemedicine approach to DR screening could have its greatest impact. This is, to our knowledge, the first evaluation of the relative influence of multi-level barriers and facilitators—categorized using CFIR—on the implementation of TDRS. We examined the views of a diverse group of health care professionals involved in healthcare delivery to people with diabetes within clinics that have had experience with TDRS. Participating clinics included rural and urban sites with varying years of experience in using TDRS, as well as varying rates of screening. Further, the use of theory for collection and analysis of our qualitative data enhanced the scientific rigor of this research .
Our results highlight the interactive and multi-level nature of factors influencing the implementation of TDRS in primary care. While patient attitudes and perceptions of TDRS are important, barriers and facilitators at the clinician, clinic, and systems levels must be addressed to improve its adoption and sustainment. Classification of emergent themes into CFIR domains and rating of constructs distinguishing high- and low-performing sites provided an actionable organization of results that will facilitate the development of targeted implementation strategies to improve the use of TDRS in FQHCs, and in other settings with comparable characteristics. Similar methods can be used to gain insight into implementation of other telemedicine-based interventions in primary care centers, specifically in clinics providing for underserved populations.
Availability of data and materials
The datasets generated and/or analyzed during the current study are not publicly available due to participant confidentiality considerations. Aggregate data are available from the corresponding author on reasonable request.
Telemedicine Diabetic Retinopathy Screening
Federally Qualified Health Center
Consolidated Framework for Implementation Research
Electronic medical records
Centers for Disease Control and Prevention. National diabetes statistics report, 2020. Atlanta: Centers for Disease Control and Prevention, U.S. Dept of Health and Human Services; 2020.
Photocoagulation treatment of proliferative diabetic retinopathy. Clinical application of Diabetic Retinopathy Study (DRS) findings, DRS report number 8. The Diabetic Retinopathy Study Research Group. Ophthalmol. 1981;88(7):583–600.
Photocoagulation for diabetic macular edema. Early treatment diabetic retinopathy study report number 1. Early treatment diabetic retinopathy study research group. Arch Ophthalmol. 1985;103(12):1796–806.
Diabetic Retinopathy Clinical Research N, Elman MJ, Aiello LP, Beck RW, Bressler NM, Bressler SB, et al. Randomized trial evaluating ranibizumab plus prompt or deferred laser or triamcinolone plus prompt laser for diabetic macular edema. Ophthalmology. 2010;117(6):1064–77 e35.
Wong TY, Sun J, Kawasaki R, Ruamviboonsuk P, Gupta N, Lansingh VC, et al. Guidelines on diabetic eye care: The International Council of Ophthalmology Recommendations for screening, follow-up, referral, and treatment based on resource settings. Ophthalmology. 2018;125(10):1608–22. https://doi.org/10.1016/j.ophtha.2018.04.007.
Summary of revisions: standards of medical care in diabetes-2019. Diab Care. 2019;42(Suppl 1):S4–6. https://doi.org/10.2337/dc19-Srev01.
Flaxel CJ, Adelman RA, Bailey ST, Fawzi A, Lim JI, Vemulakonda GA, et al. Diabetic retinopathy preferred practice pattern(R). Ophthalmology. 2020;127(1):P66–P145. https://doi.org/10.1016/j.ophtha.2019.09.025.
Maclennan PA, McGwin G Jr, Heckemeyer C, Lolley VR, Hullett S, Saaddine J, et al. Eye care use among a high-risk diabetic population seen in a public hospital's clinics. JAMA Ophthalmol. 2014;132(2):162–7. https://doi.org/10.1001/jamaophthalmol.2013.6046.
Wylie-Rosett J, Basch C, Walker EA, Zybert P, Shamoon H, Engel S, et al. Ophthalmic referral rates for patients with diabetes in primary-care clinics located in disadvantaged urban communities. J Diabetes Complicat. 1995;9(1):49–54. https://doi.org/10.1016/1056-8727(94)00005-9.
Benoit SR, Swenor B, Geiss LS, Gregg EW, Saaddine JB. Eye care utilization among insured people with diabetes in the U.S., 2010-2014. Diabetes Care. 2019;42(3):427–33. https://doi.org/10.2337/dc18-0828.
Scanlon PH. The English National Screening Programme for diabetic retinopathy 2003-2016. Acta Diabetol. 2017;54(6):515–25. https://doi.org/10.1007/s00592-017-0974-1.
Cavallerano AA, Cavallerano JD, Katalinic P, Blake B, Rynne M, Conlin PR, et al. A telemedicine program for diabetic retinopathy in a Veterans Affairs Medical Center--the Joslin Vision Network Eye Health Care Model. Am J Ophthalmol. 2005;139(4):597–604. https://doi.org/10.1016/j.ajo.2004.10.064.
Conlin PR, Fisch BM, Orcutt JC, Hetrick BJ, Darkins AW. Framework for a national teleretinal imaging program to screen for diabetic retinopathy in Veterans Health Administration patients. J Rehabil Res Dev. 2006;43(6):741–8. https://doi.org/10.1682/JRRD.2005.08.0146.
Gibson DM. Estimates of the percentage of US adults with diabetes who could be screened for diabetic retinopathy in primary care settings. JAMA Ophthalmol. 2019;137(4):440–4. https://doi.org/10.1001/jamaophthalmol.2018.6909.
Horton MB, Silva PS, Cavallerano JD, Aiello LP. Clinical components of telemedicine programs for diabetic retinopathy. Curr Diabetes Rep. 2016;16(12):129. https://doi.org/10.1007/s11892-016-0813-8.
van Eijk KN, Blom JW, Gussekloo J, Polak BC, Groeneveld Y. Diabetic retinopathy screening in patients with diabetes mellitus in primary care: Incentives and barriers to screening attendance. Diabetes Res Clin Pract. 2012;96(1):10–6. https://doi.org/10.1016/j.diabres.2011.11.003.
Leese GP, Boyle P, Feng Z, Emslie-Smith A, Ellis JD. Screening uptake in a well-established diabetic retinopathy screening program: the role of geographical access and deprivation. Diabetes Care. 2008;31(11):2131–5. https://doi.org/10.2337/dc08-1098.
Bressler NM, Varma R, Doan QV, Gleeson M, Danese M, Bower JK, et al. Underuse of the health care system by persons with diabetes mellitus and diabetic macular edema in the United States. JAMA Ophthalmol. 2014;132(2):168–73. https://doi.org/10.1001/jamaophthalmol.2013.6426.
Hartnett ME, Key IJ, Loyacano NM, Horswell RL, Desalvo KB. Perceived barriers to diabetic eye care: qualitative study of patients and physicians. Arch Ophthalmol. 2005;123(3):387–91. https://doi.org/10.1001/archopht.123.3.387.
Graham-Rowe E, Lorencatto F, Lawrenson JG, Burr JM, Grimshaw JM, Ivers NM, et al. Barriers to and enablers of diabetic retinopathy screening attendance: a systematic review of published and grey literature. Diabet Med. 2018;35(10):1308–19. https://doi.org/10.1111/dme.13686.
Ramchandran RS, Yilmaz S, Greaux E, Dozier A. Patient perceived value of teleophthalmology in an urban, low income US population with diabetes. PLoS One. 2020;15(1):e0225300. https://doi.org/10.1371/journal.pone.0225300.
Paksin-Hall A, Dent ML, Dong F, Ablah E. Factors contributing to diabetes patients not receiving annual dilated eye examinations. Ophthalmic Epidemiol. 2013;20(5):281–7. https://doi.org/10.3109/09286586.2013.789531.
Huang OS, Zheng Y, Tay WT, Chiang PP, Lamoureux EL, Wong TY. Lack of awareness of common eye conditions in the community. Ophthalmic Epidemiol. 2013;20(1):52–60. https://doi.org/10.3109/09286586.2012.751429.
Lian J, McGhee SM, Gangwani RA, Lam CLK, Yap MKH, Wong DSH. Awareness of diabetic retinopathy and its association with attendance for systematic screening at the public primary care setting: a cross-sectional study in Hong Kong. BMJ Open. 2018;8(4):e019989. https://doi.org/10.1136/bmjopen-2017-019989.
Moreton RBR, Stratton IM, Chave SJ, Lipinski H, Scanlon PH. Factors determining uptake of diabetic retinopathy screening in Oxfordshire. Diabet Med. 2017;34(7):993–9. https://doi.org/10.1111/dme.13350.
Bastos de Carvalho A, Ware SL, Lei F, Bush HM, Sprang R, Higgins EB. Implementation and sustainment of a statewide telemedicine diabetic retinopathy screening network for federally designated safety-net clinics. PLoS One. 2020;15(11):e0241767.
Liu Y, Torres Diaz A, Benkert R. Scaling up teleophthalmology for diabetic eye screening: opportunities for widespread implementation in the USA. Curr Diabetes Rep. 2019;19(9):74. https://doi.org/10.1007/s11892-019-1187-5.
Ogunyemi O, George S, Patty L, Teklehaimanot S, Baker R. Teleretinal screening for diabetic retinopathy in six Los Angeles urban safety-net clinics: final study results. AMIA Ann Symp Proc AMIA Symp. 2013;2013:1082–8.
Piyasena M, Murthy GVS, Yip JLY, Gilbert C, Zuurmond M, Peto T, et al. Systematic review on barriers and enablers for access to diabetic retinopathy screening services in different income settings. PLoS One. 2019;14(4):e0198979. https://doi.org/10.1371/journal.pone.0198979.
Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the Consolidated Framework for Implementation Research. Implement Sci. 2016;11:72.
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50. https://doi.org/10.1186/1748-5908-4-50.
Liu Y, Zupan NJ, Swearingen R, Jacobson N, Carlson JN, Mahoney JE, et al. Identification of barriers, facilitators and system-based implementation strategies to increase teleophthalmology use for diabetic eye screening in a rural US primary care clinic: a qualitative study. BMJ Open. 2019;9(2):e022594. https://doi.org/10.1136/bmjopen-2018-022594.
Mwangi N, Bascaran C, Ramke J, Kipturgo M, Kim M, Ng’ang’a M, et al. Peer-support to increase uptake of screening for diabetic retinopathy: process evaluation of the DURE cluster randomized trial. Trop Med Health. 2020;48(1):1. https://doi.org/10.1186/s41182-019-0188-z.
Riordan F, Racine E, Phillip ET, Bradley C, Lorencatto F, Murphy M, et al. Development of an intervention to facilitate implementation and uptake of diabetic retinopathy screening. Implement Sci. 2020;15(1):34. https://doi.org/10.1186/s13012-020-00982-4.
Sapru S, Berktold J, Crews JE, Katz LJ, Hark L, Girkin CA, et al. Applying RE-AIM to evaluate two community-based programs designed to improve access to eye care for those at high-risk for glaucoma. Eval Prog Plann. 2017;65:40–6. https://doi.org/10.1016/j.evalprogplan.2017.06.006.
Francis JJ, Johnston M, Robertson C, Glidewell L, Entwistle V, Eccles MP, et al. What is an adequate sample size? Operationalising data saturation for theory-based interview studies. Psychol Health. 2010;25(10):1229–45. https://doi.org/10.1080/08870440903194015.
Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88. https://doi.org/10.1177/1049732305276687.
Fielding NF. Linking data. Newbury Park: Sage; 1998.
Damschroder LJ, Lowery JC. Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implementation Sci. 2013;8(1):51. https://doi.org/10.1186/1748-5908-8-51.
King ES, Moore CJ, Wilson HK, Harden SM, Davis M, Berg AC. Mixed methods evaluation of implementation and outcomes in a community-based cancer prevention intervention. BMC Public Health. 2019;19(1):1051. https://doi.org/10.1186/s12889-019-7315-y.
Hudson KG, Lawton R, Hugh-Jones S. Factors affecting the implementation of a whole school mindfulness program: a qualitative study using the consolidated framework for implementation research. BMC Health Serv Res. 2020;20(1):133. https://doi.org/10.1186/s12913-020-4942-z.
Liang S, Kegler MC, Cotter M, Emily P, Beasley D, Hermstad A, et al. Integrating evidence-based practices for increasing cancer screenings in safety net health systems: a multiple case study using the Consolidated Framework for Implementation Research. Implement Sci. 2016;11:109.
Hodge LM, Turner KM. Sustained implementation of evidence-based programs in disadvantaged communities: a conceptual framework of supporting factors. Am J Community Psychol. 2016;58(1-2):192–210. https://doi.org/10.1002/ajcp.12082.
Keenum Z, McGwin G Jr, Witherspoon CD, Haller JA, Clark ME, Owsley C. Patients’ adherence to recommended follow-up eye care after diabetic retinopathy screening in a publicly funded county clinic and factors associated with follow-up eye care use. JAMA Ophthalmol. 2016;134(11):1221–8. https://doi.org/10.1001/jamaophthalmol.2016.3081.
Scott Kruse C, Karem P, Shifflett K, Vegi L, Ravi K, Brooks M. Evaluating barriers to adopting telemedicine worldwide: a systematic review. J Telemed Telecare. 2018;24(1):4–12. https://doi.org/10.1177/1357633X16674087.
Lau R, Stevenson F, Ong BN, Dziedzic K, Treweek S, Eldridge S, et al. Achieving change in primary care--causes of the evidence to practice gap: systematic reviews of reviews. Implement Sci. 2016;11:40.
Niazkhani Z, Pirnejad H, van der Sijs H, Aarts J. Evaluating the medication process in the context of CPOE use: the significance of working around the system. Int J Med Inform. 2011;80(7):490–506. https://doi.org/10.1016/j.ijmedinf.2011.03.009.
Tucker A, Edmonson A. Managing routine exceptions: a model of nurse problem solving behavior. In: Advances in health care management 3. Bingley: Emerald Group Publishing Limited; 2002. p. 87–113.
Bouskill K, Smith-Morris C, Bresnick G, Cuadros J, Pedersen ER. Blind spots in telemedicine: a qualitative study of staff workarounds to resolve gaps in diabetes management. BMC Health Serv Res. 2018;18(1):617. https://doi.org/10.1186/s12913-018-3427-9.
Spear SJ, Schmidhofer M. Ambiguity and workarounds as contributors to medical error. Ann Intern Med. 2005;142(8):627–30. https://doi.org/10.7326/0003-4819-142-8-200504190-00011.
Turner KM, Nicholson JM, Sanders MR. The role of practitioner self-efficacy, training, program and workplace factors on the implementation of an evidence-based parenting intervention in primary care. J Prim Prev. 2011;32(2):95–112. https://doi.org/10.1007/s10935-011-0240-1.
Varsi C, Ekstedt M, Gammon D, Ruland CM. Using the consolidated framework for implementation research to identify barriers and facilitators for the implementation of an internet-based patient-provider communication service in five settings: a qualitative study. J Med Internet Res. 2015;17(11):e262. https://doi.org/10.2196/jmir.5091.
Chasan JE, Delaune B, Maa AY, Lynch MG. Effect of a teleretinal screening program on eye care use and resources. JAMA Ophthalmol. 2014;132(9):1045–51. https://doi.org/10.1001/jamaophthalmol.2014.1051.
Stevenson L, Ball S, Haverhals LM, Aron DC, Lowery J. Evaluation of a national telemedicine initiative in the Veterans Health Administration: factors associated with successful implementation. J Telemed Telecare. 2018;24(3):168–78. https://doi.org/10.1177/1357633X16677676.
Klein KJ, Conn AB, Sorra JS. Implementing computerized technology: an organizational analysis. J Appl Psychol. 2001;86(5):811–24. https://doi.org/10.1037/0021-9010.86.5.811.
Ohinmaa A. What lessons can be learned from telemedicine programmes in other countries? J Telemed Telecare. 2006;12(Suppl 2):S40–4.
Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, et al. Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2010;(3):CD005470.
Shaw EK, Howard J, West DR, Crabtree BF, Nease DE Jr, Tutt B, et al. The role of the champion in primary care change efforts: from the State Networks of Colorado Ambulatory Practices and Partners (SNOCAP). J Am Board Fam Med. 2012;25(5):676–85. https://doi.org/10.3122/jabfm.2012.05.110281.
Davies P, Walker AE, Grimshaw JM. A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implement Sci. 2010;5(1):14. https://doi.org/10.1186/1748-5908-5-14.
We thank the participating FQHCs for their collaboration and the participating health care professionals who dedicated their time to our interviews. We thank Dr. Hayden Bosworth, Dr. Heather M. Bush, and Dr. Jing Li for their insightful suggestions on study design.
This study was funded by NCATS UL1TR001998, the University of Kentucky College of Medicine Institutional Physician Scientist Career Development Program; the Diabetes Research Center at Washington University in St. Louis of the National Institutes of Health under award number P30DK020579; and the Cincinnati Eye Institute Foundation Ignite Award (2019 Ed). The content is solely the responsibility of the authors and does not necessarily represent the official views of any funders.
Ethics approval and consent to participate
All study procedures were reviewed and approved by the University of Kentucky’s Institutional Review Board. Informed written consent was obtained from each participant prior to data collection.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Additional file 1.
Standards for Reporting Qualitative Research checklist.
Additional file 2.
Additional file 3.
Complete interview structure.
Additional file 4.
Manifestation of CFIR constructs.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Bastos de Carvalho, A., Lee Ware, S., Belcher, T. et al. Evaluation of multi-level barriers and facilitators in a large diabetic retinopathy screening program in federally qualified health centers: a qualitative study. Implement Sci Commun 2, 54 (2021). https://doi.org/10.1186/s43058-021-00157-2
- Barriers and facilitators
- Multi-level factors
- Federally Qualified Health Centers
- Primary care
- Diabetes care