Skip to main content
Download PDF
Download ePub

Preparing for the spread of patient-reported outcome (PRO) data collection from primary care to community pharmacy: a mixed-methods study

Implementation Science Communications volume 3, Article number: 29 (2022) Cite this article

Abstract

Background

Medication non-adherence is a significant public health problem. Patient-reported outcomes (PROs) offer a rich data source to facilitate resolution of medication non-adherence. PatientToc™ is an electronic PRO data collection software originally implemented at primary care practices in California, United States (US). Currently, the use of standardized PRO data collection systems in US community pharmacies is limited. Thus, we are conducting a two-phase evaluation of the spread and scale of PatientToc™ to US Midwestern community pharmacies. This report focuses on the first phase of the evaluation. The objective of this phase was to prepare for implementation of PatientToc™ in community pharmacies by conducting a pre-implementation developmental formative evaluation to (1) identify potential barriers, facilitators, and actionable recommendations to PatientToc™ implementation and (2) create a draft implementation toolkit.

Methods

Data collection consisted of demographics, observations, audio-recorded contextual inquiries, and semi-structured interviews with staff (e.g., primary care providers, pharmacists, pharmacy technicians) and patients during 1-day site visits to a purposive sample of (1) primary care practices currently using PatientToc™ and (2) community pharmacies in Indiana, Wisconsin, and Minnesota interested in the future use of PatientToc™. Post-visit site observation debriefs were also audio-recorded. Verbatim transcripts of all recordings were coded using deductive/inductive approaches and intra-/inter-site summaries were produced identifying potential barriers, facilitators, and actionable recommendations mapped to the Consolidated Framework for Implementation Research constructs. A stakeholder advisory panel engaged in an Evidence-Based Quality Improvement (EBQI) implementation process. This included “member checking” and prioritizing findings, and feedback on the adapted PatientToc™ application, implementation strategies, and accompanying toolkit for community pharmacy implementation.

Results

Two primary care practices, nine pharmacies, and 89 individuals participated. Eight major themes (four barriers and four facilitators) and 14 recommendations were identified. Throughout the four EBQI sessions, the panel (1) confirmed findings; (2) designated high priority recommendations: (a) explain PatientToc™ and its benefits clearly and simply to patients, (b) ensure patients can complete questionnaires within 10 min, and (c) provide hands-on training/resources for pharmacy teams; and (3) provided feedback on the adapted PatientToc™ application and finalized toolkit items for initial community pharmacy implementation.

Conclusions

Adoption of electronically captured PROs in community pharmacies is warranted. The implementation strategies systematically developed in this study can serve as a model for implementation of technology-driven health information patient care services, in the understudied context of community pharmacies.

Peer Review reports

Background

Medication non-adherence is a significant public health problem. Specifically, in older adults, medication non-adherence has been associated with all-cause hospitalization and mortality [1]. Community pharmacists are well-positioned to identify and intervene on medication non-adherence during medication counseling and other routine pharmacy patient care services [2]. Medication non-adherence may be influenced by negative medication-related outcomes that patients experience [3]. Patient-reported outcome (PRO) measures have become a standard in assessing patients’ behaviors towards treatment (e.g., medication adherence) [4]. Many PRO measures (e.g., Adherence Starts with Knowledge 20, General Medication Adherence Scale, Medication Adherence Reasons Scale) for medication non-adherence exist [5]. However, systematically collecting, documenting, tracking, and analyzing PROs can be problematic, particularly in the community pharmacy setting due to implementation challenges (e.g., meeting patient language needs, data privacy and security concerns, timely accessibility of data for pharmacist interventions) and limited use of standardized PRO data collection systems. Likewise, electronic patient-reported outcome data software addressing these challenges are more commonly used in other healthcare settings [4]; however, use in community pharmacy settings remains limited [6].

PatientToc™ is one type of PRO data collection software. It was initially developed by L.A. Net Community Health Resources Network investigators in California to meet the unique needs of diverse patients served by primary care practices [7]. The software facilitates PRO data collection using Android devices, with audio assist available in more than 200 languages [7]. A more detailed description and screenshots of PatientToc™ can be found in our study protocol paper [8]. The spread of this PRO data collection software from use in primary care to community pharmacies is being evaluated using a two-phase (pre-implementation followed by implementation) approach. The intended medication adherence-related PROs (i.e., responses to Brief Medication Questionnaire and Merck Medication Adherence Estimator) to be collected via PatientToc™ and plans for their use in phase two of the evaluation are provided in our study protocol paper [8]. The objectives of the study’s first phase, presented here, were to prepare for implementing PatientToc™ in community pharmacies by conducting a pre-implementation developmental formative evaluation to (1) identify potential barriers and facilitators, and actionable recommendations, to PatientToc™ implementation, and (2) create a draft implementation toolkit.

Methods

Design overview

The research team applied a convergent parallel, qualitatively driven mixed-methods [9] study design. This design equipped researchers to investigate expected barriers, facilitators, and actionable recommendations for PatientToc™ implementation in community pharmacies. Qualitative methods were the primary methods used for data collection and analysis. We intentionally provide extensive details of methods used to establish qualitative rigor (i.e., “trustworthiness”) in terms of credibility, transferability, dependability, and confirmability as defined with supporting accessible examples by Thomas and colleagues [10]. Quantitative methods were used to contextualize the study population and qualitative findings. Qualitative and quantitative data were analyzed concurrently and subsequently triangulated [11] via synthesis documents and group discussions (described in detail in the “Data synthesis” section). Reporting is in accordance with the Good Reporting of a Mixed Methods Study (GRAMMS) criteria (see Additional file 1) [12].

Conceptual frameworks

The study design was guided by three conceptual frameworks:

  1. 1.

    Curran et al.’s approach to Evidence-Based Quality Improvement (EBQI) [13]

  2. 2.

    Consolidated Framework for Implementation Research (CFIR) [14]

  3. 3.

    Expert Recommendations for Implementing Change (ERIC) framework [15]

Application of framework #1

EBQI is a multilevel process to systematically incorporate scientific findings into healthcare settings [16, 17]. This process is motivated and facilitated by researcher and local stakeholder partnerships [16, 17]. Specific to this study, we adopted Curran’s two-step approach to EBQI:

  1. 1.

    Diagnosis of site-specific implementation needs, barriers, and facilitators (i.e., formative evaluation)

  2. 2.

    The use of multi-disciplinary teams of local staff, implementation experts, and clinical experts to interpret diagnostic data and develop/adapt site-specific interventions

EBQI also operates as an implementation strategy by enabling contextual adaptation of interventions and creating buy-in among stakeholders.

Application of framework #2

We applied the CFIR to guide qualitative data collection/analysis and data synthesis. The CFIR is a well-established determinant implementation framework that is comprehensive and well suited for complex, multilevel interventions. CFIR categorizes implementation constructs across five domains: intervention characteristics, outer setting, inner setting, characteristics of the individuals involved, and the process of implementation [14].

Application of framework #3

To further classify our actionable recommendations for PatientToc™ implementation, we applied Waltz et al.’s ERIC taxonomy of implementation strategies. The taxonomy suggests 73 discrete implementation strategies grouped into nine categories or “types” of strategies: use evaluative and iterative strategies, provide interactive assistance, adapt and tailor to context, develop stakeholder interrelationships, train and educate stakeholders, support clinicians, engage consumers, utilize financial strategies, and change infrastructure [15].

Setting, site recruitment, and sampling

To address study objectives, the study setting consisted of Western primary care practices and Midwestern community pharmacies in the United States (US). The American Academy of Family Physicians defines primary care practice in the US as

the patient’s entry point into the health care system and as the continuing focal point for all needed health care services. Primary care practices are generally located in the community they serve, thereby facilitating access to health care while maintaining a wide variety of specialty and institutional consultative and referral relationships for specific care needs. The primary care practice structure often includes a team of physicians and other health professionals [18].

In addition to dispensing prescriptions (medications prescribed by an authorized provider), community pharmacists in the US are the most accessible healthcare professional to the public. Furthermore, US community pharmacies offer other pharmacy services including immunizations, disease state, and medication therapy management services. The community pharmacy structure typically includes licensed pharmacists and pharmacy technicians.

A purposive sample (n = 11) of study sites was recruited primarily through practice-based research networks (PBRN) across four states in the US: (1) (California) L.A. Net Community Health Resource Network (L.A. Net) [19], (2) Medication Safety Research Network of Indiana (Rx-SafeNet) [20], (3) Minnesota Pharmacy PBRN (MPPBRN) [21], and (4) select community pharmacies in Wisconsin. A description of each PBRN can be found in our protocol paper [8]. Typical recruitment practices (e.g., emails, phone calls) of each PBRN (mirrored for Wisconsin) were followed. To better understand how PatientToc™ has been implemented in primary care, we recruited two L.A. Net primary care practices in California, with varied approaches to PatientToc™ implementation. Three community pharmacies, consisting of a wide range of practice types and settings (e.g., urban vs. rural) from each of the remaining three states (nine total), were recruited to better understand potential barriers, facilitators, and recommendations for spreading PatientToc™ to community pharmacies.

Qualitative data collection and analysis

A core team of investigators (OAAO, GMC, HAJ, MES) with qualitative research expertise participated in qualitative data collection activities across all study sites. Additional trained investigators (LAH, NS) participated in qualitative data collection at their respective state study sites. All researchers engaged in qualitative data collection participated in pre- and onsite study-specific training. Qualitative data collection consisted of audio-recorded semi-structured interviews, investigator observations of staff and patients at participating primary care and pharmacy sites, and contextual inquiries. Observations were conducted to provide insights into how PatientToc™ is currently used (primary care) and could be used (pharmacies). Contextual inquiries consisted of informal interviews with participants demonstrating and describing elements of their work duties (see Additional file 2). Lastly, to be reflexive of the research process, summary recorded debriefs following each site visit and resulting data were included as part of our formal data collection. We also debriefed (not part of data collection) after all site visits for each state were conducted. Our protocol paper provides additional details of each qualitative data collection method [8]. As mentioned previously, CFIR informed qualitative data collection. Specifically, semi-structured interview and observation guides were designed to explore participant experiences and probe for expected barriers, facilitators, and recommendations within the CFIR constructs. Contextual inquiry probes were designed to further explore specific routine tasks associated with PatientToc™ use by primary care stakeholders as well as pharmacy tasks expected to be influenced by future PatientToc™ use. By engaging with stakeholders as they completed routine tasks, the probes further elucidated expected barriers, facilitators, and recommendations.

Data collection forms were pilot tested and refined prior to use. Specifically, primary care data collection forms were pilot tested for content and process flow by three volunteers (one provider, one staff member, and one patient) familiar with PatientToc™. Pharmacy data collection forms were pilot tested for content and process flow by two volunteers (one community pharmacist and one community pharmacy patient) not familiar with PatientToc™. Minor modifications were made to all data collection forms for clarification purposes.

A purposive sample, targeting five clinicians/staff and five patients (18 years of age or older having at least one chronic condition for which they routinely take medication) from participating primary care and pharmacy practice sites, was invited to engage with researchers during 1-day site visits.

The qualitative analysis team included two trained student research assistants and an investigator (LAH). Audio-recorded semi-structured interviews, investigator observation summary debriefs, and contextual inquiries were transcribed verbatim (InfraWare Inc, Terre Haute, IN) and checked for accuracy. The qualitative analysis team coded transcripts using NVivo 12 Pro (QSR International). Both deductive (using CFIR) and inductive (emergent from the data) approaches were used in developing the codebook. The CFIR Codebook Template [22] was adapted to fit the context of our study and frame code definitions and coding inclusion/exclusion criteria. The final codebook is available in the supplemental files (see Additional file 3).

We detail our qualitative data coding steps below:

  1. 1.

    Deductive “level 1” broad codes consisted of the 41 CFIR constructs included in the CFIR Codebook Template [22].

    1. a.

      Of note, several publications report 39 CFIR constructs [23]; however, we included all 41 listed in the CFIR Guide Codebook Template [22] in our codebook. Additional details regarding this approach to code development are provided in the supplemental files (see Additional file 3).

  2. 2.

    A total of 123 deductive sub-codes, termed “level 2” sub-codes, were included to further delineate each “level 1” broad code into potential barriers, facilitators, and actionable recommendations for PatientToc™ implementation in community pharmacies.

  3. 3.

    Subsequently, after the qualitative analysis team “level 2” coded transcripts, 234 inductive “level 3” codes were created to further delineate “level 2” codes in response to the actual data that were coded.

To illustrate this coding process, we will use the CFIR construct “Relative Advantage” as an example. Our “level 1” broad code was RelAdvantage. “Level 2” sub-codes were RelAdvantage_B, RelAdvantage_F, RelAdvantage_R, with B, F, and R indicating barrier, facilitator, and recommendation, respectively. Lastly “level 3” inductive sub-codes were created in response to the actual “level 2” coded data and included inductive codes such as Prefer paper_B, Easier_alternative_F, Increase_efficiency_R. Codes were modified, created, or collapsed as necessary [24].

Rotating pairs of analysts independently coded and reconciled the same transcripts for two study sites, until all transcripts for these sites were reconciled. Subsequently, analysts independently coded an approximately equal number of different transcripts for the remaining study sites. No double coding (i.e., using more than one code to code the same pieces of text) was permitted. Three other investigators (OAAO, HAJ, MES) with expertise in qualitative research reviewed approximately one-third of all coded transcripts and met with the qualitative analysis team regularly to provide feedback and ensure the codebook was applied consistently.

Quantitative data collection and analysis

Quantitative data collection consisted of the study site and participant demographics (see Additional file 2). Study site demographics, e.g., type of practice, number and type of staff members, prescription volume, were self-reported from the primary site contact and collected by telephone in advance of the 1-day site visits. Participant demographics, e.g., age, role, race, years employed at practice site, were self-reported and collected at the end of each participant’s interview. All quantitative data were collected and managed using Research Electronic Data Capture (REDCap™) [25] electronic tools hosted at the Indiana Clinical and Translational Sciences Institute. REDCap™ is a secure, Web-based application designed to support data capture for research studies, providing (1) an intuitive interface for validated data entry; (2) audit trails for tracking data manipulation and export procedures; (3) automated export procedures for seamless data downloads to common statistical packages; and (4) procedures for importing data from external sources [25]. Site and participant demographics were summarized by computing descriptive statistics (counts and percentages, means, and standard deviations) at the site level, and then per site descriptive statistics were used to create a mean across all sites using SPSS v. 25 (IBM Corp).

Data synthesis

Intra-site summary documents were created to summarize qualitative and quantitative findings for each participating primary care practice and pharmacy. Mixing and triangulation [11] consisted of the qualitative analysis team noting convergence or salient differences in qualitative findings by qualitative method used (e.g., if investigator observation summary debrief data confirmed or differed from interview data) and select quantitative variables, site, and participant type. Informed by intra-site summary documents, an inter-site summary document was created to facilitate investigator synthesis and identification of final themes during planned research team discussions. During the course of 2 days (13 h total), research team members met virtually (due to COVID-19 travel restrictions) via videoconferencing software to review intra- and inter-site synthesis documents and identify themes by CFIR construct. The principal investigator (MES) compiled and summarized notes from the research team discussions to identify overarching major themes, categorized as barriers or facilitators and recommendations mapped to applicable CFIR constructs. Research team members who participated in the 2-day group discussions had opportunities to review the major themes/recommendations and accuracy of mapping to CFIR constructs. Minor modifications (e.g., re-mapping of a few CFIR constructs) were made through the team review process. Mixing and triangulation [11] consisted of noting convergence or salient differences in final major themes by type of practice and participant role.

EBQI process and draft implementation toolkit

Resulting major themes informed the EBQI process for this study. This process consisted of assembling a multi-stakeholder advisory panel. In addition to a subset of study team members, the target number of Advisory Panel members was nine stakeholders—a pharmacist, a technician, and a patient from each state (i.e., one participant of each stakeholder type representing each pharmacy study site). In selecting stakeholders to invite, we attempted to balance demographics including gender, race, number of medications taken, and frequency of pharmacy visits (patients), as well as expected contributions/willingness to share ideas (based on interview responses), and perceived engagement with the site/project (for pharmacists/staff, based on site visits). For this phase of the study, we held a total of four, 120-min virtual EBQI sessions via videoconferencing software. To gather the unique perspectives of pharmacy staff and patients, “breakout” groups were held during the sessions to discuss priority questions relevant to each stakeholder group. Their insights were then shared with the whole group. The first three sessions consisted of member checking and prioritizing findings. Panelists’ initial recommendations from these sessions informed (1) mockup of the adapted PatientToc™ application and (2) draft toolkit resources for initial spread of PatientToc™ in community pharmacies. These items were reviewed during the fourth EBQI session and panelist recommendations for PatientToc™ adaptations and implementation were finalized.

Results

Keeping the study objective at the forefront and for confidentiality purposes (given small sample sizes), we first present a brief description of the current PatientToc™ implementation context at participating primary care practices (n=2) followed by a summary of quantitative, qualitative, and synthesis (barriers and facilitators) findings from participating pharmacies (n=9). Finally, we provide the actionable recommendations and the implementation toolkit items drafted in response and created during the EBQI process.

PatientToc™ implementation at participating primary care practices

Across the two primary care practices, a total of two providers (e.g., doctor, nurse practitioner), six support staff (e.g., registered nurses, medical assistants), and two patients participated. Both primary care practices serve a diverse patient population including Hispanic/Latino/Latinx/Spanish, Black/African-Americans, socioeconomically disadvantaged, medically underserved, Medicaid or dual Medicaid/Medicare beneficiaries, adult and pediatric patients, and pregnant women receiving pre-natal care. PatientToc™ implementation varied across the participating practices. One practice used it to collect survey data from Consumer Assessment of Healthcare Providers and Systems (CAHPS®), a federal program that administers surveys to capture and evaluate patient experiences, and use of the tablets has become well integrated into the facility’s workflow. Currently, the PatientToc™ tablets are mounted on rolling carts and patients’ complete questionnaires in private after being “roomed” and waiting for their provider. Many of the primary care providers and support staff are familiar with the tablet. The practice also implemented competitive incentives associated with the number of CAHPS® completed on PatientToc™.

The other practice currently stations PatientToc™ tablets in the waiting area at dedicated tables but signs indicate that tablets should not be used by patients unless instructed by staff. We did not observe tablet use by staff or patients during our visit. At this site, PatientToc™ is being used to capture “Staying Healthy Assessments,” which are part of the required California Medicaid (Medi-Cal) Initial Health Assessment. Questions focus on topics such as pain, depression, and fall risk. One L.A. Net staff member is embedded in the practice to assist with PatientToc™ implementation. While some of the primary care providers were familiar with PatientToc™, support staff appeared to have minimal familiarity.

Quantitative, qualitative, and synthesis findings from participating pharmacies

Table 1 summarizes pharmacy and participant characteristics at the site level and by participant type. A total of nine pharmacies participated in this pre-implementation phase of the study. The sites consisted of independent (n=6, 67%) and health system (n=3, 33%) pharmacies. In the US, independent pharmacies are privately held retail pharmacies not owned or operated by a publicly traded company and have no affiliation with any chain of pharmacies. For the purposes of this study, we utilized the National Council for Prescription Drug Programs (NCPDP) definition of independent pharmacy, namely one to three pharmacy locations under common ownership [26]. Likewise, a health system pharmacy is a retail pharmacy that is affiliated with a health system, which is defined as an organization that includes at least one group of providers who provide primary and/or specialty care that is integrated with each other and the hospital through joint management or common ownership. Across all sites, the mean (standard deviation (SD)) weekly prescription volume was 1266 (605) and had a mean (SD) of 3 (1) full-time equivalent pharmacists and 6 (5) full-time equivalent staff. All pharmacies offered 90-day prescription fills as a service to facilitate patients’ adherence to prescription medications. Across the nine community pharmacies, a total of 22 pharmacists and 28 pharmacy staff (e.g., pharmacy technicians, service clerks) participated and reported the mean (SD) weekly percent of hours spent working with patients as 80% (15) and 84% (19), respectively. A total of 34 pharmacy patients participated in the study. Pharmacy patients participants primarily identified as non-Hispanic White (mean (SD) 88% (22)), half (50% (30)) identified as male, 31% (37) visited their pharmacy at least once a week, and regularly used eight medications on average.

Table 1 Characteristics of participating community pharmacies (n=9), pharmacists (n=22), pharmacy staff (n=28), and patients (n=34)a
Full size table

Table 2 lists major themes (and associated theme description, representative quotations, and CFIR constructs) categorized as expected barriers and facilitators for PatientToc™ implementation in community pharmacies. We identified a total of 8 major themes: four barriers and four facilitators, for PatientToc™ implementation in community pharmacies. Convergence of qualitative results was evident across all qualitative data collection methods (semi-structured interviews and contextual inquiries with participant and investigator observation debriefs); thus, all illustrative quotations are from qualitative semi-structured interview data and included nuances by select quantitative variables including site type [(1) independent pharmacies, (2) health system pharmacies, (3) both pharmacy types, (4) primary care, (5) all primary care and pharmacy types] and participant type [(1) pharmacy staff (including staff and pharmacists), (2) pharmacy patients, (3) primary care staff (including providers and staff), (4) primary care patients, (5) primary care and pharmacy staff, (6) primary care and pharmacy staff and patients].

Table 2 Major themes, descriptions, and participant quotations categorized as potential barriers and facilitators for PatientToc™ Implementationa
Full size table

Of the four barrier-related themes, two included potential PatientToc™ integration challenges with existing pharmacy technology and within workflow. The other two barrier-related themes included potential challenges with uptake by certain patients due to concerns with data security and preferences (e.g., those who prefer providing information on paper compared to using technology). Of the four facilitator-related themes, three included pharmacy staff and leadership factors including pharmacy staff members’ willingness to try PatientToc™ if it can potentially improve patient care, respected leadership and strong communication across pharmacy team members (supporting PatientToc™ implementation), and pharmacy-related measures aligning with those expected to be impacted by PatientToc™. The last facilitator-related theme was the perceived ease of PatientToc™ use by both pharmacy staff and patients.

Recommendations and development of implementation toolkit items

A total of 14 actionable recommendations were identified (Table 3). The multi-stakeholder advisory panel was comprised of community pharmacists (n=3), pharmacy staff (n=2), and patient (n=3) participants representing seven (77.8%) of the participating pharmacy sites. Throughout three EBQI sessions, the multi-stakeholder panel confirmed the thematic findings and indicated the highest priority recommendations: (a) provide clear and simple instructions to patients that emphasize the expected benefits of PatientToc™, (b) ensure patients can complete questionnaires within 10 min, and (c) provide hands-on training/resources for pharmacy teams. Toolkit items were drafted in response. In the fourth EBQI session, panelists reviewed a mock-up of the PatientToc™ adapted for use in community pharmacies. Toolkit items for initial community pharmacy implementation were finalized. Four recommendations pertaining to intervention adaptations will be addressed in the final PatientToc™ build during phase two. Table 3 lists all of the 14 recommendations made by type of participant (pharmacy staff, patient, both) and associated ERIC implementation strategies [15] and describes specific strategy/toolkit resources.

Table 3 Summary of recommendations, implementation strategiesa, and initial PatientToc™ implementation toolkit developed through the EBQI processb
Full size table

Across the 14 actionable recommendations, there was alignment with one or more of the 9 ERIC implementation strategy categories. Of the 14 recommendations, four aligned with Engage consumers:

  • Provide clear and simple messaging to patients that emphasize the expected benefits of PatientToc™ to patients

  • Implement PatientToc™ first with specific patient sub-groups (e.g., complex patients)

  • Ensure patients can complete PatientToc™ questionnaires within 2 to 10 min (e.g., pre-populate information when possible, reduce need for typing)

  • Use PatientToc™ to optimize patient prescription wait times as well as other appointment-based services (e.g., MTM, medication synchronization program)

Accompanying implementation toolkit items included patient-facing print materials (large and small posters, pamphlets, bag stuffers), scripted language for pharmacies’ use, sample workflows, and workflow “cheat sheets.”

Three recommendations aligned with Adapt and tailor to context:

  • Work with pharmacy teams and vendors to ensure PatientToc™ is well integrated with the pharmacies’ dispensing systems

  • Consider adapting PatientToc™ for more languages (mockup/demo was provided in English only)

  • Consider including patient education and/or information/referrals to pharmacy services on PatientToc™

These will primarily be addressed as part of the intervention development/finalized build of the adapted PatientToc™ application; however, a “Referrals Cheat Sheet” implementation toolkit item was created to provide a modifiable template of resources for pharmacy teams to consider when reviewing PatientToc™ results and addressing medication non-adherence. Of note, Work with pharmacy teams and vendors to ensure PatientToc™ is well integrated with the pharmacies’ dispensing systems was one of the most extensive adaptations to the PatientToc™ application and we plan to include this capability in the adapted finalized build during phase two.

Discussion

Engaging key stakeholders from multiple perspectives provided invaluable insight into resources and strategies needed for initial implementation of PatientToc™ in community pharmacies. These collaborative efforts positioned us to prioritize actionable recommendations, refine and tailor an adapted PatientToc™ design, and develop an initial implementation toolkit unique to the community pharmacy context, which ultimately is expected to result in more effective spreading and scaling efforts. To our knowledge, this is the first study to evaluate electronic collection and use of electronic PRO data in community pharmacy settings using implementation science approaches. As stakeholders’ insights were the foundation of this pre-implementation evaluation, we focus our discussion on stakeholders’ highest priority recommendations. As well, we situate the findings in the implementation science and pharmacy practice literatures and discuss policy-related implications.

The first two high priority recommendations, (1) provide clear and simple instructions to patients that emphasize the expected benefits of PatientToc™ and (2) ensure that patients can complete PatientToc™ questionnaires within 2 to 10 min, align with the ERIC implementation strategy “Engage Consumers: Prepare Patients/Consumers to be Active Participants” [15]. The first recommendation regarding the emphasis of expected benefits to patients has practice and policy-related implications. For example, in the US, medication therapy management (MTM) under Medicare Prescription Drug policy is a service often provided by community pharmacists [27]; however, the literature indicates lack of MTM uptake by some beneficiaries may be due to gaps in patient-friendly communication and marketing on MTM and its benefits [28]. Our toolkit items and specific strategies for this recommendation, including patient-facing print materials and scripted language for pharmacy team members, align with suggestions cited in the pharmacy practice literature [28] and could serve as a model for promoting adoption of other pharmacy services and related policies.

Similar to the second high priority recommendation of using more brief questionnaires, Rolstad et al. [29] review and meta-analysis findings suggested questionnaire response rates were statistically lower for longer questionnaires; however, the researchers argue that the impact of content should not be overlooked, further emphasizing the importance of buy-in of patients, regardless of questionnaire length. Our toolkit items and marketing and messaging strategies include benefits, specifically to patients, expected from completing PatientToc™ questionnaires. Time to complete the questionnaires will be examined during phase two (implementation) of this study.

The third and final high priority recommendation, provide hands-on training and resources for pharmacy teams, possibly for continuing education credit, to support PatientToc™ implementation, aligns with several ERIC implementation strategies in the “Train and Educate Stakeholders” category, including develop and distribute education materials and work with educational institutions. Our training toolkit items align with these strategies, particularly our Social Determinants of Health Continuing Education modules (per request of advisory panel members) offered in collaboration with a college of pharmacy continuing education office. This recommendation was not surprising, as implementation strategies focusing on training and educating stakeholders are ubiquitous. For example, Thoele et al. describe their intervention implementation process in an acute care hospital setting. Researchers developed 54 toolkit items over a three-phase process. Of these 54 toolkit items, 25 (46%) were related to training medical staff during the first two phases. Training and education strategies are often described as necessary for implementation efforts, but they are usually not sufficient on their own [30]; hence, our implementation toolkit is composed of a variety of implementation strategies spanning the nine ERIC implementation strategy categories.

Implementation science is an iterative process; thus, we are taking a similar approach and aim to do two plan-do-study-act cycles in the next phase of this study to refine our implementation toolkit for spreading and scaling PatientToc™ to community pharmacies. We anticipate revisions to the pharmacy team’s training modules may be required after initial implementation and prior to further scaling of PatientToc™.

Limitations

While this study has several strengths, findings should be considered in the context of its limitations. First, only Midwestern and no chain community pharmacies were recruited in this phase of the study. However, we made great efforts to account for nuances in findings by pharmacy and participant type and we provide participating pharmacies’ demographic data, which can aid in transferring findings to different community pharmacy settings with similar characteristics. Nevertheless, we will attempt to recruit chain community pharmacies in the scale-up phase of evaluation and will revise implementation strategies as needed.

Although comprehensive, we applied a complex approach to data analysis by including all 41 constructs of CFIR, which ultimately resulted in over 200 inductive sub-codes. This approach was time consuming and required a change in our analysis approach from analysts independently coding and reconciling the same transcripts for all study sites to two sites with the remaining transcripts being divided across analysts to be independently coded with frequent research team member quality checks. Furthermore, results could have varied if a different implementation framework was used. However, CFIR proved to be useful as it addressed our study objectives, facilitated the identification of key implementation barriers and facilitators, and helped elicit recommendations, which corresponded to several widely recognized implementation strategies [15]. Lastly, the developmental formative evaluation approach used in this study prioritized qualitative methods to explore potential barriers, facilitators, and actionable recommendations for implementing PatientToc™ in community pharmacies. A more quantitative approach, such as a survey, could better capture frequencies of potential barriers, facilitators, and recommendations. However, using qualitative approaches, such as semi-structured interviews, was preferred for our goal of gaining in-depth perspectives from participants rather than a generalized understanding [31]. Thus, using both methods allowed access to data and interpretations of findings that each method alone could not provide.

Conclusions

Adoption of health information technology, specifically to electronically capture PROs in the community pharmacy setting, is warranted. Applying implementation science methods and strategies can aid in adopting such interventions. If shown to be effective, the implementation strategies systematically developed in this study can serve as models for implementing other health information technology-driven, patient care services in the understudied context of community pharmacies.

Availability of data and materials

The datasets generated and/or analyzed during the current study are not publicly available because individuals’ privacy could be compromised. However, reasonable requests for deidentified datasets can be made to the corresponding author for consideration.

Abbreviations

CAHPS®:

Consumer Assessment of Healthcare Providers and Systems

CFIR:

Consolidated Framework for Implementation Research

EBQI:

Evidence-Based Quality Improvement

ERIC:

Expert Recommendations for Implementing Change

GRAMMs:

Good Reporting of a Mixed Methods Study

L.A.:

Los Angeles

MTM:

Medication therapy management

PROs:

Patient-reported outcomes

REDCap:

Research Electronic Data Capture

SD:

Standard deviation

References

  1. Walsh CA, Cahir C, Tecklenborg S, Byrne C, Culbertson MA, Bennett KE. The association between medication non-adherence and adverse health outcomes in ageing populations: a systematic review and meta-analysis. Br J Clin Pharmacol. 2019;85(11):2464–78. https://doi.org/10.1111/bcp.14075.

    Article  PubMed  PubMed Central  Google Scholar 

  2. CDC: Tailored pharmacy-based interventions to improve medication adherence. https://www.cdc.gov/dhdsp/pubs/medication-adherence.htm. Accessed 21 Jan 2022.

  3. Gellad WF, Grenard JL, Marcum ZA. A systematic review of barriers to medication adherence in the elderly: looking beyond cost and regimen complexity. Am J Geriatr Pharmacother. 2011;9(1):11–23. https://doi.org/10.1016/j.amjopharm.2011.02.004.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Meirte J, Hellemans N, Anthonissen M, Denteneer L, Maertens K, Moortgat P, et al. Benefits and disadvantages of electronic patient-reported outcome measures: systematic review. JMIR Perioper Med. 2020;3(1):e15588. https://doi.org/10.2196/15588.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Kwan YH, Weng SD, Loh DHF, Phang JK, Oo LJY, Blalock DV, et al. Measurement properties of existing patient-reported outcome measures on medication adherence: systematic review. J Med Internet Res. 2020;22(10):e19179. https://doi.org/10.2196/19179.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Wuyts J, Maesschalck J, De Wulf I, De Lepeleire J, Foulon V. Studying the impact of a medication use evaluation by the community pharmacist (Simenon): patient-reported outcome measures. Res Soc Adm Pharm. 2020;16(12):1760–7. https://doi.org/10.1016/j.sapharm.2020.01.017.

    Article  Google Scholar 

  7. PatientToc. www.patienttoc.com. Accessed 21 Jan 2022.

  8. Snyder ME, Chewning B, Kreling D, Perkins SM, Knox LM, Adeoye-Olatunde OA, et al. An evaluation of the spread and scale of PatientToc™ from primary care to community pharmacy practice for the collection of patient-reported outcomes: a study protocol. Res Social Adm Pharm. 2020. https://doi.org/10.1016/j.sapharm.2020.03.019.

  9. Hesse-Biber S. Qualitative approaches to mixed methods practice. Qual Inq. 2010;16(6):455–68. https://doi.org/10.1177/1077800410364611.

    Article  Google Scholar 

  10. Thomas E, Magilvy JK. Qualitative rigor or research validity in qualitative research. J Spec Pediatr Nurs. 2011;16(2):151–5. https://doi.org/10.1111/j.1744-6155.2011.00283.x.

    Article  PubMed  Google Scholar 

  11. Greene JC, Caracelli VJ, Graham WF. Toward a conceptual framework for mixed-method evaluation designs. Educ Eval Policy Anal. 1989;11(3):255–74. https://doi.org/10.3102/01623737011003255.

    Article  Google Scholar 

  12. O'Cathain A, Murphy E, Nicholl J. The quality of mixed methods studies in health services research. J Health Serv Res Policy. 2008;13(2):92–8. https://doi.org/10.1258/jhsrp.2007.007074.

    Article  PubMed  Google Scholar 

  13. Curran GM, Mukherjee S, Allee E, Owen RR. A process for developing an implementation intervention: QUERI series. Implement Sci. 2008;3(1). https://doi.org/10.1186/1748-5908-3-17.

  14. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50. https://doi.org/10.1186/1748-5908-4-50.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, Chinman MJ, Smith JL, et al. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implement Sci. 2015;10(1):109. https://doi.org/10.1186/s13012-015-0295-0.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Goldstein KM, Vogt D, Hamilton A, Frayne SM, Gierisch J, Blakeney J, et al. Practice-based research networks add value to evidence-based quality improvement. Healthc (Amst). 2018;6(2):128–34. https://doi.org/10.1016/j.hjdsi.2017.06.008.

    Article  Google Scholar 

  17. Rubenstein LV, Meredith LS, Parker LE, Gordon NP, Hickey SC, Oken C, et al. Impacts of evidence-based quality improvement on depression in primary care: a randomized experiment. J Gen Intern Med. 2006;21(10):1027–35. https://doi.org/10.1111/j.1525-1497.2006.00549.x.

    Article  PubMed  PubMed Central  Google Scholar 

  18. AAFP: Primary care. https://www.aafp.org/about/policies/all/primary-care.html. Accessed 21 Jan 2022.

  19. L.A. Community health resource network. http://www.lanetpbrn.net. Accessed 21 Jan 2022.

  20. Purdue University College of Pharmacy. Medication Safety Research Network of Indiana. http://www.pharmacy.purdue.edu/rx-safenet. Accessed 21 Jan 2022.

  21. Minnesota Pharmacy Practice Based Research Network. https://cdn.ymaws.com/www.mpha.org/resource/resmgr/imported/PBRN/index.html. Accessed 21 Jan 2022.

  22. CFIR Tools and Templates - Data analysis tools: codebook template. https://cfirguide.org/tools/tools-and-templates/. Accessed 21 Jan 2022.

  23. Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the Consolidated Framework for Implementation Research. Implement Sci. 2016;11(1):72. https://doi.org/10.1186/s13012-016-0437-z.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Bradley EH, Curry LA, Devers KJ. Qualitative data analysis for health services research: developing taxonomy, themes, and theory. Health Serv Res. 2007;42(4):1758–72. https://doi.org/10.1111/j.1475-6773.2006.00684.x.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–81. https://doi.org/10.1016/j.jbi.2008.08.010.

    Article  PubMed  Google Scholar 

  26. Independent pharmacies in the U.S. are more on the rise than on the decline. https://www.pcmanet.org/wp-content/uploads/2020/03/FINAL_Independent-Pharmacies-in-the-U.S.-are-More-on-the-Rise-than-on-the-Decline.pdf. Accessed 21 Jan 2022.

  27. Adeoye OA, Farley JF, Coe AB, Pestka DL, Farris KB, Zillich AJ, et al. Medication therapy management delivery by community pharmacists: insights from a national sample of Medicare part D beneficiaries. J Am Coll Clin Pharm. 2019;2(4):373–82. https://doi.org/10.1002/jac5.1160.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Ferreri SP, Hughes TD, Snyder ME. Medication therapy management: current challenges. Integr Pharm Res Pract. 2020;9:71–81. https://doi.org/10.2147/IPRP.S179628.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Rolstad S, Adler J, Rydén A. Response burden and questionnaire length: is shorter better? A review and meta-analysis. Value Health. 2011;14(8):1101–8. https://doi.org/10.1016/j.jval.2011.06.003.

    Article  PubMed  Google Scholar 

  30. Wensing M, Fluit C, Grol R. Educational strategies. Improving patient care: the implementation of change in health care. Hoboken: Wiley; 2013. p. 197–209.

    Book  Google Scholar 

  31. Moser A, Korstjens I. Series: practical guidance to qualitative research. Part 3: sampling, data collection and analysis. Eur J Gen Pract. 2018;24(1):9–18. https://doi.org/10.1080/13814788.2017.1375091.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We are grateful to the study sites and individual participants as well as the many students and colleagues who assisted with data collection, analysis, and preparation of the adapted PatientToc™ and implementation toolkit items. We thank Lyndee Knox, Chief Executive Officer of PatientToc™, for assisting with site recruitment from L.A. Net primary care practices and participating in the Evidence-Based Quality Improvement process.

Funding

This research was supported by grant number R18HS025943 (PI: Snyder) from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality. Dr. Adeoye-Olatunde was supported by the Indiana Clinical and Translational Sciences Institute funded in part by award number TL1TR001107 (A. Shekhar, PI) from the National Institutes of Health, National Center for Advancing Translational Sciences, Clinical and Translational Sciences Award, outside the work submitted. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. A portion of Dr. Curran’s salary is supported by a grant (UL1TR003107) awarded to the Translation Research Institute at the University of Arkansas for Medical Sciences from the National Center for Advancing Translational Sciences (NCATS) of the National Institutes of Health (NIH).

Author information

Authors and Affiliations

  1. Purdue University College of Pharmacy, 640 Eskenazi Avenue, Fifth Third Bank Building, Indianapolis, IN, 46202, USA

    Omolola A. Adeoye-Olatunde, Heather A. Jaynes & Margie E. Snyder

  2. University of Arkansas for Medical Sciences College of Pharmacy, 4301 W. Markham St., #522-4, Little Rock, AR, 72205-7199, USA

    Geoffrey M. Curran

  3. University of Minnesota, College of Pharmacy, 308 Harvard Street, SE, Minneapolis, MN, 55455, USA

    Lisa A. Hillman & Jon C. Schommer

  4. University of Wisconsin-Madison School of Pharmacy, 777 Highland Ave., Madison, WI, 53704, USA

    Nisaratana Sangasubana, Betty A. Chewning & David H. Kreling

  5. Purdue University College of Pharmacy, 575 Stadium Mall Drive, Heine Pharmacy Building, West Lafayette, IN, 47907, USA

    Matthew M. Murawski

  6. Indiana University School of Medicine, 410 W 10th Street, Indianapolis, IN, 46202, USA

    Susan M. Perkins

Authors
  1. Omolola A. Adeoye-Olatunde
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Geoffrey M. Curran
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Heather A. Jaynes
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lisa A. Hillman
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nisaratana Sangasubana
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Betty A. Chewning
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. David H. Kreling
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jon C. Schommer
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Matthew M. Murawski
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Susan M. Perkins
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Margie E. Snyder
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

OAAO methodology, validation, formal analysis, investigation, resources, data curation, writing the original draft, writing, review, editing, visualization, supervision, and project administration. GMC conceptualization, methodology, validation, formal analysis, investigation, resources, writing, review, editing, supervision, project administration, and funding acquisition. HAJ conceptualization, methodology, software, validation, formal analysis, investigation, resources, data curation, writing, review, editing, visualization, supervision, project administration, and funding acquisition. LAH validation, formal analysis, investigation, resources, data curation, writing, review, editing, and visualization. NS investigation, resources, data curation, writing, review, and editing. BAC conceptualization, methodology, validation, formal analysis, writing, review, editing, supervision, and funding acquisition. DK conceptualization, formal analysis, writing, review, editing, and funding acquisition. JCS conceptualization, formal analysis, writing, review, editing, supervision, and funding acquisition. MMM conceptualization, writing, review, editing, and funding acquisition. SMP conceptualization, methodology, writing, review, editing, and funding acquisition. MES conceptualization, methodology, validation, formal analysis, investigation, resources, data curation, writing, review, editing, visualization, supervision, project administration, and funding acquisition. All authors read and approved the final manuscript and have agreed both to be personally accountable for the author’s own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature.

Corresponding author

Correspondence to Omolola A. Adeoye-Olatunde.

Ethics declarations

Ethics approval and consent to participate

This study (#1901010204) was granted exemption by the Indiana University Human Subjects Office, Office of Research Compliance.

Consent for publication

Not applicable.

Competing interests

MES served as a consultant to Westat, Inc. on an evaluation of the CMS Enhanced MTM program from 2016 to 2020. All other authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Good Reporting of A Mixed Methods Study (GRAMMS) checklist.

Additional file 2.

Data collection forms.

Additional file 3.

Codebook.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Adeoye-Olatunde, O.A., Curran, G.M., Jaynes, H.A. et al. Preparing for the spread of patient-reported outcome (PRO) data collection from primary care to community pharmacy: a mixed-methods study. Implement Sci Commun 3, 29 (2022). https://doi.org/10.1186/s43058-022-00277-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s43058-022-00277-3

Keywords