Implementing point-of-care medical information systems into trauma and general surgeon practice in a middle-income country: a qualitative study utilizing the Consolidated Framework for Implementation Research
Implementation Science Communications volume 4, Article number: 38 (2023)
Point-of-care medical information systems (POCMIS) can provide an efficient and effective means of strengthening health systems globally through their encouragement of continued medical education. Using the Consolidated Framework for Implementation Research (CFIR) as a guide, this research provides suggestions for improved implementation of POCMIS in low- and middle-income countries informed by an intervention implemented across public and military hospitals in Lima, Peru.
Analysis is based on qualitative interviews conducted with 12 Peruvian surgeons across eight public hospitals and one military hospital who received an intervention that provided free access to UpToDate and introduced Google Translate. The post-intervention interviews were transcribed, translated, and analyzed for themes overlapping with CFIR constructs to expose barriers to implementation and suggestions for improved implementation of future interventions.
Barriers included a lack of seniority buy-in and engaged leadership, an overabundance of personal preferences for multiple POCMIS, and a culture of assumption that inhibited open communication regarding access to and use of POCMIS. Suggestions for improved implementation focused on the adaptation of the intervention. Namely, surgeons discussed regionally-specific adaptations as well as adaptations specific to their surgical specialty including visual, rather than written, representation of the information available via POCMIS.
Results indicate necessary adaptations for implementing interventions including POCMIS in LMICs, mimicking much of the implementation science literature on intervention adaptation. In addition to explicit suggestions provided by surgeons, we also suggest actionable steps to adapt to barriers identified in our data. Rapid assessment procedures (RAP) are one established methodological technique useful for assessing organization culture prior to implementation, allowing for necessary cultural adaptations. Dynamic adaption process (DAP) is another useful and established method that breaks implementation into four phases allowing for adaptations based on the initial assessment of the intervention site.
Quality improvement of surgical care in low- and middle-income countries (LMICs) has been a World Health Organization priority since establishing the Global Initiative for Emergency and Essential Surgery Care in 2005 . Given the reported link between the use of information systems encouraging evidence-based medicine and improved patient outcomes , such information systems are of interest across resource settings. One cost-effective path to global health equity is building strong surgical systems [3, 4], and continued medical education (CME) has the potential to strengthen surgical systems. Information systems such as UpToDate (UTD) support CME, suggesting a pathway to stronger surgical systems [5,6,7].
With medical literature constantly evolving as new research is published, providers often cannot stay current for more than a few years after completing medical school . Evidence from one systematic review shows that increased years of provider practice correlates with lower performance—measured as clinical knowledge and adherence to standards of practice—suggesting providers are less likely to follow updated clinical guidelines the further they are from their postgraduate medical education. This evidence held true across subspecialties, including surgery .
Safely applying primary literature to clinical practice is time-consuming . When time is limited, point-of-care medical information systems (POCMIS) may help by efficiently and effectively encouraging CME [5, 11, 12]. POCMIS refer to technological and informational tools that assist providers with clinical decision-making . Studies investigating POCMIS across medical specialties show high use of and satisfaction with POCMIS in healthcare settings globally  along with improvement in self-directed e-learning for CME .
Despite these known benefits, financial and language barriers to POCMIS remain for healthcare providers in non-English speaking countries and LMICs . Other barriers discussed in the literature include lack of: time, awareness of accessible resources, and information intake capacity, to name a few . In response to these persisting barriers, this short report provides recommendations for how we can encourage effective implementation of POCMIS for surgeons in LMICs.
The educational intervention for this research included a didactic on evidence-based practice (EBP) and use of Google Translate, along with provision of application for free UTD access to surgical providers at nine hospitals in Lima, Peru. Participants were interviewed after receiving the intervention and their responses were analyzed for this short report, which conforms to the consolidated criteria for reporting qualitative research (COREQ [See Additional file 1]).
Surgeons practicing in the USA (L.N.L.), Peruvian General Surgery Society board members (M.R., G.B., D.O.C., J.H.), a medical educator (J.L.R.), and graphic designer (W.L.R.) developed a one-hour presentation reviewing theories of EBP from previously published EBP courses [18,19,20,21,22, 22,23,24], used interactive clinical practice questions alongside UTD articles, and introduced Google Translate. The study team invited 12 of the largest hospitals from the military, public, and social security systems in Lima to participate. Nine of these hospitals ultimately enrolled . After the presentation, participants applied for a grant to receive free, individual access to UTD for one calendar year via the Better Evidence UTD Donations Program . Upon closure of data collection, the control group also received the intervention.
Following the intervention, we conducted qualitative interviews with 10 attending surgeons and two residents. The interview sample consisted of eight general surgeons, two trauma surgeons, one laparoscopic surgeon, and one combined general surgeon/surgical oncologist. Of the 12 providers, three identified as women and nine as men.
We used the Consolidated Framework for Implementation Research (CFIR) to create a semi-structured interview guide focused on how surgeons find information to answer their clinical questions and about their experiences using UTD (see Additional file 2). Interviews were conducted via phone by a Peruvian researcher with an extensive practical background in qualitative research, lasted an average of 41 min, and were recorded on the interviewer’s laptop for transcription and translation by a bilingual researcher.
Data were uploaded into Dedoose Version 7.0.23 (Sociocultural Research Consultants, Los Angeles, California) for coding and analysis by H.E.N. and M.W. following procedures outlined by Braun and Clarke . Transcripts were open coded, coders were blind to each other’s coding, and differences were resolved by discussion until 100% agreement was reached. A second qualitative methodologist (S.N.W.) further analyzed specific code excerpts from the initial analysis to compile findings for this report. See Additional file 3 for detailed methods.
Findings from the interviews expose barriers to implementation and provide suggestions for improved implementation of the POCMIS intervention. We present those barriers and suggestions using constructs identified in CFIR (18 [see Appendix for CFIR diagram]).
Barriers to implementation
Thematically, barriers to implementation emerged as three separate themes in the data: seniority buy-in and engaged leadership, personal preference, and culture of assumption. Seniority buy-in and engaged leadership relates to the construct of implementation climate within the hospitals. Personal preference relates to CFIR’s other personal attributes construct. Culture of assumption is explained through CFIR constructs culture and networks and communication. See Table 1 of Additional File 4 for a summary on how barriers identified correlate with CFIR constructs, themes from the analysis, and particularly illustrative quotes from the interviews.
Seniority buy-in and engaged leadership
Assumptions specific to senior surgeons and those with authority came through in the interviews as a motivator for action—or inaction—and in doing so exposed implementation climate barriers. For example, about a previous chief of service at their hospital, one surgeon said,
It was different a few years ago when Doctor [redacted] was the chief of service, he was constantly looking up new studies, new information. He guided us and gave us orientation, he motivated us to keep looking up information. We lost all of that since he retired. (INT-01)
Another surgeon further demonstrated the impact individuals have on organizational culture and medical practice when discussing another senior physician, telling us,
I read, double check, and follow what evidence-based medicine says, but unfortunately I live in an “obedience-based medicine” system…my boss says “you have to do this with the patient,” “but doctor, this drainage is not recommended,” “no, you have to do it,” “but literature has shown meta-analysis where this doesn’t work on certain patients,” “I don’t care, you have to do it, I have more experience with many more patients.” (INT-09)
In the same way, an engaging chief of service can motivate colleagues to continue learning and growing as surgeons, a leader resistant to change can impede growth. As yet another surgeon put it,
Sometimes the senior workers are more reluctant to change…They don’t accept the changes that come with technology. It’s a constant battle. It gets a little tedious because they take it personally. (INT-04)
Notably, the same interviewee who commented on an association between seniority and reluctance to change acknowledged the value of collaboration with more experienced partners in certain clinical scenarios: “If I’m dealing with a difficult case, I usually turn to more experienced people who can give me some advice on how to solve the problem.” (INT-04).
When discussing surgeons’ preferences during interviews, we were met with a long list of personal preferences for a variety of search engines, including UTD, PubMed, Google, YouTube, and more. For instance, one surgeon told us, “it’s up to each other’s preferences. Some are still using the regular system (unintelligible), others have PubMed. It depends on what they like” (INT-10). Providers having pre-established preferences for POCMIS made universal adoption of the intervention challenging.
Culture of assumption
Cultural barriers and networks and communication barriers came through in the data as one theme: culture of assumption. For example, one surgeon told us “I think it’s useful to have this technology handy. If they didn’t have the motivation to use it before, they have it now, I believe they have it now.” (INT-08). When followed up with a question about what search engines colleagues use, the surgeon then said, “we have talked about it when [the researcher] came here, she did a presentation, but I couldn’t say,” referring to the presentation from the intervention.
This surgeon’s response indicates a common theme among interviewees’ assumptions about POCMIS culture among their colleagues, i.e., a culture of assumption, rather than a culture of discussion. Take the following interview excerpt:
P: ...They say “ok, I’m going to do it!,” but I don’t ask them later if they did it or not. It’s usually a conversation in the moment and I don’t do a follow up later, like “hey, did you check this?, did you login on UpToDate?” None of that.
I: You didn’t overhear any comments if they did it either?
P: Not really. We haven’t talked about that. (INT-07).
Suggestions for improved implementation
In addition to demonstrating barriers to implementation, our interviews with surgeons also revealed practical suggestions for improved implementation. We present explicit suggestions in the results section while extrapolating further suggestions from the data in the discussion section. See Table 2 in Additional file 4 for a summary on how suggestions correlate with CFIR constructs, themes from the analysis, and particularly illustrative quotes from the interviews.
Adaptability was a prominent area for suggested improvements throughout the data. Suggestions for improved adaptability touch on two areas: regionally-specific adaptations and specialty-specific adaptations.
Regionally specific adaptations
Adaptations related to the specific region of practice are vital based on interview responses. For example, one surgeon told us, “Since we’re here in Lima, we do have resources to diagnose patients, but we fall short when it’s about treatment. We don’t have too many resources.” (INT-09). Similarly, another surgeon mentioned how the evidence they find through POCMIS may not be applicable to their region:
Since they are taken from literature produced in other countries, I’ve found out about other treatment options and even though it doesn’t apply to what I do here, now I know about new and different treatments I could use if they were available here, mostly in the surgical field. (INT-07)
One participant also noted how financial resources differ by region, saying “probably in the United States [UTD is] affordable, but here it’s a lot of money” (INT-03).
Specialty specific adaptations
Participants also noted their desire for more visual presentation of information via POCMIS, rather than written information with one surgeon, saying “compared to other medical specialties…we go for another type of information: graphic and visual information” (INT-06). Another surgeon told us, “Solutions for surgical problems are not written down and I can tell you that [from] experience” (INT-04). Surgeons also noted the uniqueness of the production of medical knowledge in surgical specialties saying, “experiments cannot be performed on human beings and…a complex surgery performed in an animal cannot be extrapolated to a human being” (INT-09).
The research presented here aims to provide a roadmap for future interventions focused on increasing access to POCMIS for surgeons in LMICs. Because CFIR was developed in a high-income context, applying it to LMICs often requires adaptation . In line with that literature, our results highlight the importance of adaptability for low-resource settings. Other researchers have similarly noted the need to adapt surgery-focused interventions—as well as the frameworks used for implementation—for LMICs, such as Zambia, Brazil, and Benin [29, 30].
Also important in our findings were specialty-specific adaptations, which took surgeons’ desires for visual and graphic information into account. With attention to the growing literature on adaptation in implementation science, we suggest our findings on adaptability of this intervention are used to help build an adaptome data platform  to better understand how deviations from fidelity may lead to better outcomes based on context. Though many surgeons requested more visuals which are easily accessible with limited time, the user experience of UTD was generally positive with participants reporting plans to use UTD in the future as well as positive feedback regarding its ease of use.
Suggestions for encouraging seniority buy-in and engaged leadership
In addition to explicit recommendations from surgeons themselves, we suggest necessary steps for ensuring seniority buy-in and engaged leadership during implementation. Such efforts may also positively impact the implementation climate by fostering a learning climate. Two major components of a learning climate—including leaders’ expressing fallibility and a need for colleagues’ assistance, and team members feeling they are a valuable part of the change process—were not clearly identified in the implementation climate. In line with previous literature on implementation , ensuring those in leadership roles and with organizational influence support the intervention is an essential first step in ensuring success. Further, considering theories of technology uptake in healthcare settings may be necessary to address these barriers.
Suggestions for adapting to cultural barriers
In carefully choosing leaders with both formal and informal influence in an organization to champion an intervention, we recognize the power of individuals as carriers of culture and their influence on implementation climate . In doing so, we also recognize culture as an enabling force in addition to the constraining force it can be .
To harness culture as the enabling force it has the potential to be, we suggest future implementations begin with rapid assessment procedures (RAP) to assess the organizational culture . As an ethnographic method, RAP is well-situated to assess culture, including organizational culture, by using methods such as interviews with key informants and organizational stakeholders. Other researchers have successfully used RAP for implementing health informatics, such as electronic health records  and computerized provider order entry , in healthcare settings. While researchers have identified limitations of RAP , the benefits of introducing RAP to implementation efforts in healthcare settings far outweigh those limitations .
Another option for adapting to cultural barriers is the dynamic adaptation process (DAP)—a four-phased process for intervention adaptation that balances intervention fidelity with adaptability . In adopting this approach, implementors can assess organizational culture in phase one of DAP, helping them determine what adaptations will be necessary to prepare prior to implementation.
Toward the end of the study, COVID-19 spread across the world, requiring our team to finish data collection virtually. Therefore, it is reasonable to assume our sample included less representation of ideas across surgeons involved in the intervention. In other words, participants may have been less likely to participate in the qualitative interviews because the global pandemic may have shifted their priorities. Also reasonable is the assumption that participants were rightfully preoccupied by the emerging global pandemic, inhibiting their ability to fully reflect on the intervention. While the interview data we collected are rich and provide important suggestions for improved implementation, we recognize the limitations in our sample.
With a worldwide agenda for quality improvement of surgical care in LMICs, interventions will continue to be implemented across settings and cultures . Whether those interventions include POCMIS or not, cultural barriers and the adaptability of the intervention will be important factors. This research provides practical advice for improved implementation of those interventions based on one study conducted in Lima, Peru.
Availability of data and materials
The qualitative datasets generated and analyzed during the current study are not publicly available given the need for specific context to grasp their usefulness. However, code reports from the qualitative interviews are available from the corresponding author on reasonable request.
Point-of-care medical information systems
Low- and middle-income countries
Continued medical education
Consolidated Framework for Implementation Research
Peruvian General Surgery Society
Rapid assessment procedures
Dynamic adaptation process
Bickler SW, David S. Improving surgical care in low- and middle-income countries: a pivotal role for the World Health Organization. World J Surg. 2010;24:386–90. https://doi.org/10.1007/s00268-009-0273-2.
Valtis YK, Rosenberg J, Bhandari S, Wachter K, Teichman M, Beauvais S, et al. Evidence-based Medicine for all: what we can learn from a programme providing free access to an online clinical resource to health workers in resource-limited settings. BMJ Glob Health. 2016;1:e000041. https://doi.org/10.1136/bmjgh-2016-000041.
Grimes CE, Henry JA, Maraka J, Mkandawire NC, Cotton M. Cost-effectiveness of Surgery in Low- and Middle-income Countries: A Systematic Review. World J Surg. 2014;38:252–63.
Chan L, Arunachalam S, Kirsop B. Open access: a giant leap towards bridging health inequities. Bull World Health Organ. 2009;87:631–5.
World Health Organization. Monitoring the building blocks of Health Systems: a handbook of indicators and their measurement strategies. https://www.who.int/healthinfo/systems/WHO_MBHSS_2010_full_web.pdf. Accessed 21 Nov 2019.
Binagwaho A, Kyamanywa P, Farmer PE, Nuthulaganti T, Umubyeyi B, Nyemazi JP, et al. The Human Resources for Health Program in Rwanda – A New Partnership. N Engl J Med. 2013;369:2054–9.
World Health Organization. World health report 2006: working together for health. http://www.who.int/whr/2006/en. Accessed 15 Dec 2019.
Norman G, Shannon S, Marrin M. The need for needs assessment in continuing medical education. BMJ. 2004;328:999–1001.
Choudry NK, Fletcher RH, Soumerai SB. Systematic review: the relationship between clinical experience and quality of health care. Ann Intern Med. 2005;142(4):260–73.
Kyriakoulis K, Patelarou A, Laliotis A, Wan AC, Matalliotakis M, Tsiou C, et al. Educational strategies for teaching evidence-based practice to undergraduate health students: systematic review. J Educ Eval Health Prof. 2016;13:34.
Moja L, Kwag K. Point of care medical information services: a platform for self-directed continuing medical education for front line decision makers. Postgrad Med J. 2015;91:83–91.
Hudspeth J, Morse M. Health Information and Global Health Inequity: Point-of-Care Knowledge Systems as a Foundation for Progress. J Gen Intern Med. 2016;32(5):572–5.
Department of Health and Human Services. Clinical Decision Support. Available from: https://www.healthit.gov/topic/safety/clinical-decision-support. Accessed 22 Feb 2021.
Papandria D, Fisher JG, Kenney BD, Dykes M, Nelson A, Diefenback KA. Orientation of Perpetuity: An Online Clinical Decision Support System for Surgical Residents. J Surg Res. 2019;19(245):649–55.
Kulier R, Gülmezoglu AM, Zamora J, Plana MN, Carroli G, Cecatti JG, et al. Effectiveness of a Clinically Integrated e-Learning Course in Evidence-Based Medicine for Reproductive Health Training: a randomized trial. JAMA. 2012;308(21):2218–25.
LaGrone L, Fuhs A, Egoavil E, Langdale L, Guangworawong P, Hamasaki J, Gyedu A, et al. A Global Assessment of Access to and Use of Medical Information: The State of Evidence-Based Surgery. World J Surg. 2018;42:521–31.
Daei A, Soleymani MR, Ashrafi-rizi H, Zargham-Boroujeni A, Kelishadi R. Clinical information seeking behavior of physicians: A systematic review. Int J of Med Inform. 2020;139:104144. https://doi.org/10.1016/j.ijmedinf.2020.104144.
Tilson JK, Kaplan SL, Harris JL, Hutchinson A, Ilic D, Niederman R, et al. Sicily statement of classification and development of evidence-based practice learning assessment tools. BMC Med Educ. 2011;11:78.
Sackett DL. Evidence-based medicine. Semin Perinatol. 1997;21(1):3–5.
Rousseau DM, Gunia BC. Evidence-Based Practice: The Psychology of EBP Implementation. Annu Rev Psychol. 2016;68:667–92.
Stillwell SB, Fineout-Overholt E, Melnyk BM, Williamson KB. Asking the Clinical Question: A Key Step in Evidence-Based Practice. Am J Nurs. 2010;110(3):58–61.
Potisek NM, McNeal-Trice K, Barone MA. The whole “PROOF”: incorporating evidence-based medicine into clinical teaching. J Pediatr. 2017;140:e20171073.
Brownson RC, Eyler AA, Harris JK, Moore JB, Tabak R. Getting the Word Out: New Approaches for Disseminating Public Health Science. J Public Health Manag Pract. 2018;24(2):102–11.
Ubbink DT, Legemate DA, Koelemay MJ. The Merits of a Two-Day Evidence-Based Medicine Course for Surgical Residents. World J Surg. 2016;40(8):1809–14.
World Health Organization. Global Health Workforce Alliance. Peru. https://www.who.int/workforcealliance/countries/per/en/. Accessed 28 Sept 2021.
Better Evidence. Ariadne Labs.. Available from https://www.ariadnelabs.org/areas-of-work/better-evidence/. Cited 16 Dec 2019
Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101. https://doi.org/10.1191/1478088706qp063oa).
Means AR, Kemp CG, Gwayi-Chore MC, et al. Evaluating and optimizing the consolidated framework for implementation research (CFIR) for use in low- and middle-income countries: a systematic review. Implementation Sci. 2020;15:17. https://doi.org/10.1186/s13012-020-0977-0.
Chambers DA, Norton WE. The Adaptome: Advancing the Science of Intervention Adaptation. Am Jour of Prev Med. 2016;51(4):S124–31. https://doi.org/10.1016/j.amepre.2016.05.011.
White MC, Randall K, Cap-Chichi NFE, et al. Implementation and evaluation of nationwide sclae-up of the Surgical Safety Checklist. Br J Surgery. 2019;206(2):e91–102. https://doi.org/10.1002/bjs.11034.
Aarons GA, Green AE, Palinkas LA, et al. Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implementation Sci. 2012;7:32. https://doi.org/10.1186/1748-5908-7-32.
Emmons KM, Weiner B, Fernandez ME, Tu SP. Health Educ Behav. 2012;39(1):87–105.
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50. https://doi.org/10.1186/1748-5908-4-50.
Hays S. Structure and agency and the sticky problem of culture. Sociol Theory. 1994;12(1):57–72.
Harris KJ, Jerome NW, Fawcett SB. Rapid assessment procedures: a review and critique. Hum Org. 1997;56(3):375–8.
McMullen CK, Ash JS, Sittig DF, Bunce A, Guappone K, Dykstra R, Carpenter J, Richardson J, Wright A. Rapid assessment of clinical information systems in the healthcare setting: an efficient method for time-pressed evaluation. Methods Inf Med. 2011;50(4):299–307. https://doi.org/10.3414/ME10-01-0042.
Ash JS, Sittig DF, McMullen CK, Guappone K, Dykstra R, Carpenter J. A rapid assessment process for clinical informatics interventions. AMIA Annu Symp Proc. 2008;6(2008):26–30.
Holdsworth LM, Safaeinili N, Winget M, et al. Adapting rapid assessment procedures for implementation research using a team-based approach to analysis: a case example of patient quality and safety interventions in the ICU. Implementation Sci. 2020;15:12. https://doi.org/10.1186/s13012-020-0972-5.
Consolidated Framework for Implementation Research. What is CFIR? Diagram [Image on Internet]. Ann Arbor: CFIR Research Team-Center for Clinical Management Research; 2022. Available from: https://cfirguide.org
Dr. German Málaga and Dr. José Luis Rojas, professors of evidence-based medicine at Universidad Peruana Cayetano Heredia (UPCH), helped brainstorm and review didactic session materials. Mr. Walter Limaco Romero, a graphic designer at UPCH, helped creatively detail our didactic session for presentation. Ms. Marcela Quispe, a medical student at UPCH, aided in cluster 2 presentations and data collection. Ariadne Labs’ Better Evidence Grant provided our qualifying participants with funds for free one-year UpToDate access, which can be re-applied for each year. Lastly, Mercy Díaz Añazgo provided transcription and translation services for our qualitative interview data.
The Peruvian General Surgery Society Research Collaborative includes José Albinagorte, Alfredo Allahual, David Alvarez, Ricardo Arones, Gerardo Arredondo, Giuliano Borda, Jaime Herrera, Ricardo Herrera, Eduardo Huaman, Eduardo Morales, Addis Ayabar Morón, Ricardo Perez, Gustavo Tagle, Moises Valiente, Oscar Villa
The research reported here was funded by a Fogarty Fellowship awarded to HN. The fellowship was provided through the Global Health Program for Fellows and Scholars at the National Institutes of Health’s Fogarty International Center (Sponsor’s Funding ID: D43TW009345, Grants Office ID: A145171).
Ethics approval and consent to participate
This study was approved by the University of Washington and the Universidad Peruana Cayetano Heredia institutional review boards. Researchers gained informed consent per guidelines from the approving institutional review board from all participants in the study.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix: Consolidated Framework for Implementation Research Diagram 
About this article
Cite this article
Wilson, S.N., Noble, H., Ordoñez, W.J.N. et al. Implementing point-of-care medical information systems into trauma and general surgeon practice in a middle-income country: a qualitative study utilizing the Consolidated Framework for Implementation Research. Implement Sci Commun 4, 38 (2023). https://doi.org/10.1186/s43058-023-00397-4