Establishing a primary care audit and feedback implementation laboratory: a consensus study

Background There is a significant variation among individual primary care providers in prescribing of potentially problematic, low-value medicines which cause avoidable patient harm. Audit and feedback is generally effective at improving prescribing. However, progress has been hindered by research waste, leading to unanswered questions about how to include audit and feedback for specific problems and circumstances. Trials of different ways of providing audit and feedback in implementation laboratories have been proposed as a way of improving population healthcare while generating robust evidence on feedback effects. However, there is limited experience in their design and delivery. Aim To explore priorities, feasibility, and ethical challenges of establishing a primary care prescribing audit and feedback implementation laboratory. Design and setting Two-stage Delphi consensus process involving primary care pharmacy leads, audit and feedback researchers, and patient and public. Method Participants initially scored statements relating to priorities, feasibility, and ethical considerations for an implementation laboratory. These covered current feedback practice, priority topics for feedback, usefulness of feedback in improving prescribing and different types of prescribing data, acceptability and desirability of different organization levels of randomization, options for trial consent, different methods of delivering feedback, and interest in finding out how effective different ways of presenting feedback would be. After receiving collated results, participants then scored the items again. The consensus was defined using the GRADE criteria. The results were analyzed by group and overall score. Results Fourteen participants reached consensus for 38 out of 55 statements. Addressing antibiotic and opioid prescribing emerged as the highest priorities for action. The panel supported statements around addressing high-priority prescribing issues, taking an “opt-out” approach to practice consent if waiving consent was not permitted, and randomizing at lower rather than higher organizational levels. Participants supported patient-level prescribing data and further research evaluating most of the different feedback methods we presented them with. Conclusions There is a good level of support for evaluating a wide range of potential enhancements to improve the effects of feedback on prescribing. The successful design and delivery of a primary care audit and feedback implementation laboratory depend on identifying shared priorities and addressing practical and ethical considerations. Supplementary Information The online version contains supplementary material available at 10.1186/s43058-020-00103-8.


Introduction
Audit and Feedback (A&F) are one of the methods that are routinely used to improve patient care. Health care professionals, working either in a team or individually, receive feedback on their performance by reflecting on data derived from their routine practice. A&F are intended to enhance professional performance and thereby improve the quality of health care and patient safety. We previously used A&F in the Campaign to Reduce Opioid Prescribing (CROP). CROP reduced predicted opioid prescription spending by £900,000 in West Yorkshire.
We are exploring the feasibility of scaling up the CROP intervention for a larger primary care randomised controlled trial (RCT) to reduce harmful opioid prescribing whilst improving the effectiveness of A&F.
In this first stage of the consensus process we will present you with a range of prescribing priority issues, ethical and governance issues and potential modifications to A&F reports that may increase effectiveness.
Please grade each item on the scale of 1 to 9 in terms of importance, priority and usefulness as prompted by each question. There is also an 'Unable to score' response if you are unable to rate any outcome. If you require more information to answer the question please click on 'more info' to see additional information.There will also be space for you to leave comments that you feel are important to consider in our interpretation of the results.
Please click next to move to the next page to give your consent to take part in this study.  Required I understand that any information I provide, including personal details, will be confidential, stored securely and only accessed by those carrying out the study. I or my institution will not be identifiable in the report or reports that result from the research without my permission. (Initial next to the statement if you agree)  Required I agree for the data collected from me to be stored and used in relevant future research in an anonymised form. (Initial next to the statement if you agree)  Required I understand that other genuine researchers will have access to this data only if they agree to preserve the confidentiality of the information as requested in this form. (Initial next to the statement if you agree)  Required I understand that relevant sections of the data collected during the study, may be looked at by individuals from the University of Leeds or from regulatory authorities where it is relevant to my taking part in this research. I give permission for these individuals to have access to my records. Show practice achievement as a combination of graph, infographic or number Deliver paper copies of feedback reports by post Deliver copies of feedback reports by email Deliver feedback to practices with an online dashboard (not connected to patient record system) Deliver feedback to practices with an online dashboard (connected to patient record system) Deliver feedback on total prescriptions (for a particular medication) Deliver feedback on total number of patients (taking a particular medication) Deliver feedback on specific high risk groups for a particular medication Give feedback on specific behaviours to change (e.g. dose reduction or particular patients to avoid prescribing for) Deliver educational outreach or training to practices on the feedback topic Ask practices to complete action plans in response to the feedback report Ask practices to document the potential benefits to their practice and/or patients on changing behaviour Provide feedback at regular intervals for the same topic (e.g. monthly, quarterly, yearly) Provide feedback intensely at the start of a feedback cycle and then reduce frequency Provide feedback for more than one year on one topic Provide feedback for just one year on one topic Please tick all feedback options that you currently do in practice.
Please comment on your current practice in using audit and feedback to primary care practices to help us with our interpretation.
Topic for primary care prescribing feedback There are multiple high priority prescribing issues in current UK primary care.
Please score each prescribing issue identified firstly as to its importance (potential to improve patient safety and care) and then secondly its priority (based on whether there are existing interventions to improve safety and care) to tackle in an audit and feedback intervention. Please rate the priority to tackle in primary care of each of the following prescribing issues on a scale of 1 (Low priority for further interventions) to 9 (High priority for further interventions) Click 'more info' for more detail about the proposed prescribing issues.
Are there any other high priority or important prescribing issues that we should add to this list?
Audit and feedback evidence Reviews of audit and feedback trials have shown that the effects are generally moderate or small but there may be a substantial population effect. Unfortunately, the reviews also show we still have limited information with which to support decisions on how best to use audit and feedback in routine health care. Please comment on your scoring above to help us with our interpretation.

Usefulness of data
Using opioid prescribing as an example, there are different sources of primary care prescribing data that give different types of data. We would like to know which types of data would be the most useful for primary care prescribers to use to improve their opioid prescribing. Sub-groups of patients (e.g. patients at high risk of escalation of opioid doses or long term opioid use or those at higher risk of adverse drug events) Please rate the types of data on their usefulness to primary care practitioners on a scale of 1 (Not at all useful) to 9 (Very useful) If you have rated specific opioid medications as useful or important, please detail the specific medications that would be most useful and important to concentrate on.

Randomisation
We would like to know how acceptable different levels of randomisation would be in a trial of different modifications to audit and feedback.

Consent
There are ethical decisions relating to consent to taking part in a trial of different modifications to audit and feedback on opioid prescribing in primary care, e.g. consent for practice-aggregated data extraction from medical records and consenting to taking part in a randomised experiment. Please rate the following different consent options on firstly, how acceptable they are and secondly how ideal they are.
 More info Acceptability of randomisation level 1 2 3 4 5 6 7 8 9 Unable to answer/do not know Consent practices individually, asking them to sign up to a opioid prescribing feedback trial (practice opt-in) Provide practices information on the trial and allow them to withdraw from the trial if they wish (practice opt-outlikely to increase recruitment compared to opt-in)

Consent at Sustainablity and Transformation Plan level for data access
Waive consent as the burden of responding to consent request is higher than taking part in the trial (used by NHS England feedback study on antibiotic prescribing) Please rate how acceptable the potential options for consent for a trial on a scale of 1 (Not at all acceptable) to 9 (Very acceptable)  More info Ideal randomisation level 1 2 3 4 5 6 7 8 9 Unable to answer/do not know Consent practices individually, asking them to sign up to a opioid prescribing feedback trial (practice opt-in) Provide practices information on the trial and allow them to withdraw from the trial if they wish (practice opt-outlikely to increase recruitment compared to opt-in)

Consent at Clinical Commissioning Group level for data access
Consent at Sustainablity and Transformation Plan level for data access Please rate how ideal the potential options for consent for a trial on a scale of 1 (Not at all ideal) to 9 (Very ideal) Waive consent as the burden of responding to consent request is higher than taking part in the trial (used by NHS England feedback study on antibiotic prescribing)

Report delivery
We would like to know how interested you are in finding out whether the following potential methods of delivery of feedback reports to primary care practices are the most effective at improving prescribing.
Please rate each item firstly on its acceptability to practices and then secondly on how ideal it is for practices.
Acceptablility of feedback delivery methods 1 2 3 4 5 6 7 8 9 Unable to answer/do not know Provide (multiple) copies of a paper-based feedback report to each practice Send a PDF copy of the report via email to each practice Have an online dashboard that practices can log into to view their report (not linked to patient record system) Have an online dashboard that practices can log into that connects to the patient record system to identify patients where review is needed Please rate how acceptable the potential options for feedback delivery are on a scale of 1 (Not at all acceptable) to 9 (Very acceptable) Ideal feedback delivery methods 1 2 3 4 5 6 7 8 9 Unable to answer/do not know Provide (multiple) copies of a paper-based feedback report to each practice Send a PDF copy of the report via email to each practice Have an online dashboard that practices can log into to view their report (not linked to patient record system) Have an online dashboard that practices can log into that connects to the patient record system to identify patients where review is needed Please rate how ideal the potential options for feedback delivery are on a scale of 1 (Not at all ideal) to 9 (Very ideal) Please comment on your scoring above to help us with our interpretation.

Modifications to feedback reports
Even on the basis of the best evidence available, no strong recommendations can be given regarding the best way to introduce audit and feedback into routine practice. We wish to test important (but potentially subtle) variations in audit and feedback that may have important effects.
We would like to know how interested you are in finding out whether the following potential modifications to feedback reports are the most effective.
Interest in finding out if feedback modification is more effective Whether feedback identifying specific behaviours to be changed is more effective (e.g. reducing opioid drug dose escalation, reduction in patients taking combinations of opioids) Whether feedback on its own is more (cost-) effective than feedback delivered with educational outreach or training Whether asking practitioners to document the implications of changing practice is more effective (e.g. asking practitioners to document the potential benefits for their patients or their practice by reducing opioid prescriptions) Please rate how interested you would be in finding out if the following options for feedback modifications are the most effective are on a scale of 1 (Not at all interested) to 9 (Very interested) Whether the frequency or the number of times feedback is delivered affects achievement (e.g. providing quarterly feedback for 1, 2 or 3 years or whether an increased frequency (monthly) at the start of the feedback period followed by a reduction in frequency (to 6 monthly) after 1 year is more effective than regular quarterly feedback) Are there any other high priority or important A&F modifications that we should add to this list for testing?
Please comment on your scoring above to help us with our interpretation.

-What is your role Audit and Feedback Researcher Medicines Optimisation Lead Patient and Public Representative
Final page Thank you for completing this consensus questionnaire, I am very grateful for your time. I will be collating all responses and will send you the second round of this consensus questionnaire in late June with a copy of your answers and all participant answers combined. You will then have a chance to re-answer the questions based on the results.
If you have any comments regarding this questionnaire or the consensus process please email Sarah Alderson (s.l.alderson@leeds.ac.uk).