Effectiveness of confidential reports to physicians on their prescribing of antipsychotic medications in nursing homes

Background Antipsychotic medication use in nursing homes is associated with potential for harms. In Ontario, Canada, an agency of the provincial government offers nursing home physicians quarterly audit and feedback on their antipsychotic prescribing. We compared the characteristics of physicians who did and did not engage with the intervention, and assessed early changes in prescribing. Methods This population-level, retrospective cohort study used linked administrative databases to track prescribing practices in nursing homes pre-intervention (baseline), immediately post-initiative (3 months), and at follow-up (6 months). Exposure variables identified whether a physician signed up to participate (or not) or viewed the feedback following sign up (or not). Differences in the proportion of days that residents received antipsychotic medications at 6 months compared to baseline by exposure(s) were assessed using a linear mixed effects regression analysis to adjust for a range of resident, physician, and nursing home factors. Benzodiazepine and statin prescribing were assessed as a balance and tracer measures, respectively. Results Of 944 eligible physicians, 210 (22.3%) signed up to recieve the feedback report and 132 (13.9%) viewed their feedback. Physicians who signed up for feedback were more likely to have graduated from a Canadian medical school, work in urban nursing homes, and care for a larger number of residents. The clinical and functional characteristics of residents were similar across physician exposure groups. At 6 months, antipsychotic prescribing had decreased in all exposure groups. Those who viewed their feedback report had a signicantly greater reduction in antipsychotic prescribing than those who did not sign up (0.94% patient-days exposed; 95% CI 0.35 to 1.54%, p = 0.002). Trends in prescribing patterns across exposure groups for benzodiazepines and statins were not statistically significant. Interpretation Almost a quarter of eligible physicians engaged early in a voluntary audit and feedback intervention related to antipsychotic prescribing in nursing homes. Those who viewed their feedback achieved a small but statistically significant change in prescribing, equivalent to approximately 14,000 fewer days that nursing home residents received antipsychotic medications over 6 months. This study adds to the literature regarding the role of audit and feedback interventions to improve quality of care.


Background
Antipsychotic medications are commonly used in nursing homes, especially in patients with agitation and/or behavioral disturbances [1]. The potential risks of antipsychotic medications in older adults include cardiovascular events, falls, decreased cognition, and mortality [2][3][4][5][6][7]. For older residents living in nursing homes, risks of unmanaged aggressive behavior must also be considered for those living with, and caring for, the resident [8]. Therefore, the goal for clinicians, nursing homes, and health systems is not complete avoidance of antipsychotic medications but regular reassessment of the balance between risk for harms and benefits.
Health systems have attempted to encourage appropriate antipsychotic medication prescribing through a range of quality improvement strategies [3], including public reporting of potentially inappropriate antipsychotic medication prescribing in nursing homes [9,10]. These strategies are not consistently effective [2,3,[11][12][13][14]. One challenge arising in the interpretation of such evidence to inform policy is that those individuals willing to participate in trials of quality improvement strategies are not necessarily representative of the target population [15] and would benefit most from the intervention.
Herein, we describe an evaluation of the early impact of a voluntary, large-scale audit and feedback (A&F) initiative on antipsychotic medication prescribing in Ontario nursing homes. A&F works by directing recipients' attention to a gap between desired and actual practice, so that efforts can be made to close this gap. Just as pills only work for those who take them, A&F is likely to work only for those who engage with the intervention. Our objectives were to describe the extent of early engagement in this initiative across nursing home physicians; compare the characteristics of physicians, nursing homes, and residents by extent of engagement; and assess whether engagement was associated with changes in the proportion of nursing home residents receiving antipsychotics over time.

Study design
This was a population-level, retrospective cohort study of nursing home residents and their most responsible physicians in Ontario, Canada, from July 2015 to March 2016. This time period covers 3 months preceding the intervention (baseline-quarter, July to September 2015), the immediate post-intervention 3-month period (postquarter-one (Q1), October to December 2015), and next 3-month period (post-quarter-two (Q2), January to March 2016; see Fig. 1). The study received approval

Contributions to the literature
Audit and feedback is known to have effects that vary widely.
This paper evaluates a natural experiment with the launch of a province-wide audit and feedback intervention to improve prescribing in nursing homes.
It shows how these effects depend on engagement with the intervention.
It also compares physicians that engaged early-on in this voluntary audit and feedback initiative to those who did not, showing some systematic differences that could inform future work targeting clinicians whose patients are most in need of improved care.

Setting
Ontario is Canada's most populous province, with approximately 13 million people. All personal and nursing care within nursing homes in Ontario is funded by the provincial government through the Ministry of Health and Long-Term Care. Residents are responsible for accommodation charges such as room and board, the costs of which are set by the provincial government and are standard across the province. Rate reductions are available through a government subsidy for those with low income on a case-by-case basis. Prescription drug costs for nursing home residents are covered by the Ontario Drug Benefit program, if prescribed by an Ontario physician or other authorized prescriber. In nursing homes, residents typically have a most responsible physician who prescribes their medications. Day-to-day care is handled by allied health professionals including nurses and personal support workers, with ratios stipulated by provincial legislation. Health Quality Ontario, now Ontario Health (Quality), is the provincial government agency mandated to monitor and report to the public on the quality of health care provided in Ontario and to support improvements in quality. In 2015, in collaboration with Health Quality Ontario, we established an implementation science laboratory to support the optimization of A&F initiatives in Ontario [16].

Data sources
Data were obtained from administrative databases linked using encoded identifiers and analyzed at ICES. ICES is a prescribed entity in Ontario with the capacity to hold and link patient-level databases for the purposes of health system evaluation and planning (the research team could not alter these records). The databases at ICES include information on all hospital and nursing home admissions in the province, all visits to emergency departments, physician billing claims, and vital statistics, as well as prescription data for those covered under the provincial health insurance program [17][18][19][20][21]. Intervention exposure data were captured by Health Quality Ontario and shared confidentially for analysis at ICES (www.ices.on.ca). ICES is an independent, non-profit research institute whose legal status under Ontario's health information privacy law allows it to collect and analyze health care and demographic data, without consent, for health system evaluation and improvement. This project was approved by ICES' Privacy and Legal Office. It was also approved by local research ethics boards at Women's College Hospital and the University of Toronto.

Cohort development
Nursing home residents aged 66 to 105 were eligible for inclusion in the cohort if they were admitted to a nursing home in Ontario at any time between July 1, 2015, and March 31, 2016. A resident could leave and re-enter the cohort if they were discharged (i.e., for a hospitalization) and then readmitted to a nursing home at a later date within this period. Residents remained in the cohort until their discharge date, death date, or end of the observation period. The Continuing Care Reporting System-Long-Term Care was used to assess date of admission and discharge, as well as demographic, clinical, and functional data, captured through the validated Resident Assessment Instrument (RAI) [22]. A full RAI assessment completed by nursing home staff is legislatively mandated within 14 days of admission and updated annually or with a change in status; a quarterly RAI assessment is required every 92 days. For each 3month period under investigation, residents were assigned to a most responsible physician according to previously defined algorithms [10]. We excluded patients whose most responsible physicians could not benefit from the intervention due to data suppression in the feedback reports (i.e., physicians with fewer than six nursing home residents have their data suppressed for privacy reasons (due to small cell sizes)).

Baseline physician, nursing home, and resident characteristics
We extracted characteristics from the administrative databases during the baseline-quarter (i.e., July to September, 2015), using the earliest month of data available during this time period. We used the ICES physician database to assess prescriber characteristics, including sex, age, years in practice, specialty, and foreign medical graduate status. We assessed the number of residents for whom each physician was the most responsible provider (Additional file 1). We also assessed total Ontario Health Insurance Program (OHIP) billings to describe the number of nursing home claims in each time period, and the proportion of total resident assessments this represented of the physician's entire nursing home practice. For nursing homes, we used the institutional facilities database at ICES to assess nursing home characteristics, namely the number of beds, rurality, and private/public ownership status.
We used the RAI data to ascertain demographic and clinical characteristics of residents that might be associated with the outcomes of interest, including sex, age, duration of residency in the home, comorbid conditions (e.g., Alzheimer's (including other dementia), depression), and clinical assessment scores (e.g., activity of daily living scale, pain scale, depression rating score, likelihood of falls scale, aggressive behavior score). We used OHIP data to determine whether residents had a specialist consultation in the prior year by a geriatrician or psychiatrist. We also used OHIP to assess whether the resident had any physician encounters with a recorded diagnosis of psychosis in the prior 5 years. We used the Canadian Institute for Health Information (CIHI) datasets to assess whether residents had an emergency department visit in the prior year (using the National Ambulatory Care Reporting System (NACRS) database) and whether residents had a hospital admission in the prior year (using the Discharge Abstract Database (DAD)). These databases provide complete population-level data for the variables of interest.

Intervention and engagement
The Health Quality Ontario reports for physicians working in nursing homes were initially developed as part of a broader Appropriate Prescribing Demonstration Project, in partnership with the Ontario Medical Association and the provincial government [23]. The reports were developed with input from a multidisciplinary team of experts and stakeholders including nursing home physicians [24]. Health Quality Ontario uses administrative data sources to report on a series of quality indicators, and physicians across the province can sign up to receive confidential information about their practice. The reports are updated and re-released quarterly. (In this study, we examined effects related to the initial report released on September 29, 2015, and the subsequent two reports released on January 29, 2016, and April 29, 2016. See Additional file 2 for examples of the reports).
Beginning in July 2015, Health Quality Ontario promoted the reports to nursing home physicians via communication materials distributed by Health Quality Ontario and external partners (including the Ontario Long-Term Care Association, the Ontario Association of Non-Profit Home and Services for Seniors, and the Ontario Long-Term Care Clinicians). To sign up for the report, physicians had to provide consent to receive the report, and verify their email address and identity. When a new report was available for download, those who signed up would receive email notification from Health Quality Ontario. To view the report, physicians had to log into their account via Health Quality Ontario's secure web portal, and then download a PDF of the report. The steps required to engage with the reports created three natural levels of exposure for our analysis: (1) physicians who did not sign up during the study period, (2) physicians who signed up but did not view the report(s) during the study period, and (3) physicians who viewed at least one report.

Prescribing outcomes
The primary outcome was the proportion of days a resident was prescribed any antipsychotic medication. The Ontario Drug Benefit database holds complete, population-level dispensing for Ontarians living in nursing homes. For each 3-month time period analyzed (i.e., baseline, Q1, Q2), we obtained the total number of days that the resident was present in the nursing home setting (denominator) and also assessed whether they had at least one active prescription for an antipsychotic that covered those days (numerator). Similar measures were calculated for benzodiazepine prescribing (used as a balance measure to test whether initiatives to decrease antipsychotic medications might result in these high-risk sedative agents being used as an alternative) and statin prescribing (used as a "tracer" or negative control measure, to assess general trends in (de)prescribing habits unlikely to be attributable to the intervention).

Analysis
Descriptive statistics were used to examine physician, nursing home, and resident characteristics based on patterns of sign up and viewing of reports.
We used histograms to visually inspect the normality of distributions for each outcome measure. The unit of analysis was the individual resident. We used linear mixed effects regression analysis to compare the prescribing outcomes between the three groups from baseline to Q1 and Q2. For this analysis, we excluded those physicians who signed up too late to receive the initial report. The dependent variable was the percentage of nursing home days the resident had an active prescription (i.e., the days covered by the prescription divided by the days in the study period). The exposure variable was a three-level categorical variable, defined as did not sign up, signed up but did not view the report, and signed up and viewed the report. The model included a categorical variable for quarter and the interaction between exposure group and quarter. The correlation in quarterly repeated measures on the same resident was accommodated by specifying an unstructured covariance matrix. A random intercept and random period effect were specified to account for correlation amongst multiple residents nested in the same nursing home and over time.
The model adjusted for the following home-, provider-, and resident-level characteristics: number of beds, urban vs. rural location, and private vs. public nursing home; provider sex, age, years practicing, foreign vs. domestic graduate, number of nursing home residents in the practice, number of nursing homes practicing in, and proportion of OHIP billings in nursing home in comparison to all other billings; and resident sex, age, length of time in nursing home, number of Charlson comorbidities, RAI variables (including diabetes, hypertension, arteriosclerotic heart disease, chronic heart failure, peripheral vascular disease, deep vein thrombosis, cardiac dysrhythmia, dementia, cancer, obstructive air disease, depression, arthritis, Parkinson's disease), level of function (activities of daily living scale), pain score, depression rating score, likelihood of falls scale, aggressive behavior scale, frailty index, emergency department visits in past year, inpatient hospitalizations in past year, any psychiatric consult in past year, any geriatric consult in past year, and any concurrent benzodiazepine use. Adjusted least square mean differences together with 95% confidence intervals were obtained from the model to estimate differences for all variables (a) between the three exposure groups at baseline, Q1, and Q2; (b) within the three groups from baseline to Q1 and baseline to Q2; and (c) between the three exposure groups in their change from baseline to Q1 and baseline to Q2.
All analyses were conducted using SAS Version 9.4. Given the risk of type 1 error, we a priori selected a p value threshold of 0.01 to assess for statistical significance. Figure 2 describes the study flow for included patients and their most responsible nursing home physician. In each quarter (i.e., 3-month time period) under analysis, 99.3% of included residents had a unique primary physician prescriber in their nursing home.

Results
Comparing exposure groups Table 1 shows the baseline physician, nursing home, and resident characteristics by physician exposure (i.e., signup status). A total of 944 physicians met eligibility criteria for the time period of the analysis. Of the 239 physicians who ultimately signed up for the intervention, 2 were not eligible at that time, leaving 237 physicians who signed up for this comparison.
Physicians who signed up for the report were more likely to work in larger (average of 162 beds (SD 89.4)), urban nursing homes (78.4%). These physicians were more likely to have graduated medical school in Canada and tended to have a greater proportion of the practice focused on nursing home care, with a larger nursing home resident load. The average charactersistics of residents in each practice did not differ between the physicians who did and did not sign up for the report, although those who signed up had slightly greater proportions of their patient group with a history of Alzheimer's, depression, aggressive behavior, and elevated fall risk. There was no statistically significant difference in baseline antipsychotic medication prescribing rates between those who did and did not sign up to receive a report, but those who signed up prescribed  benzodiazepines and statins to a smaller proportion of their roster. Table 2 describes the same characteristics mentioned above for comparing exposure groups, focusing on those who did and did not view their reports. Of the 210 physicians who signed up in time to receive the initial intervention, 132 viewed at least 1 report and 78 did not view any of their reports. These groups were quite similar on the measured characteristics, although Canadian medical graduates viewed their reports more often than foreign medical graduates, and slightly fewer residents of physicians who viewed their reports received psychiatric consultations in the prior year. Figure 3 describes the model-adjusted output for the mean percentage of days receiving antipsychotic medications for each exposure group at baseline, Q1, and Q2, illustrating differences in prescribing over time for those physicians who viewed the reports, but not for other groups of physicians. Figure 4 depicts the modeladjusted change in prescribing at each timepoint, relative to the baseline value. Additional file 3: Figures S1 and S2 depict the model-adjusted change for the balance measure, the percentage of days on benzodiazepines, and for the tracer measure, the percentage of days on statins. Together, these figures illustrate change over time in prescribing for each group of physicians. Table 3 quantifies the changes in prescribing for these medication classes over time within each exposure group. We observed significant changes over time in all three groups for antipsychotic medications. The greatest reduction in antipsychotic medication over time was observed for the group that viewed at least one of their reports: − 1.82% (95% CI − 1.27 to − 2.37%; p < 0.0001). Table 4 summarizes the comparisons across exposure groups for changes in prescribing over time. For antipsychotic medications, there was a statistically significant difference between the group that viewed the report and the group that did not sign up at all (0.94% greater reduction; 95% CI 0.35 to 1.54%; p = 0.002). For both benzodiazepines and statins, no statistically significant changes in prescribing for these classes over time were observed when comparing those who viewed the report and those did not sign up.

Main findings
In this observational study, we found that total exposure to antipsychotic medications amongst nursing home residents declined over time and the rate of decline was associated with greater engagement in a voluntary A&F intervention. Specifically, physicians who both signed up and then viewed their personalized prescribing reports had a greater decrease in their prescribing rates than physicians who did not. The decrease in prescribing over 6 months amongst the group that viewed their A&F reports equates to approximately 14,000 fewer days that any nursing home resident was exposed to antipsychotic medications in that timeframe. In contrast, we did not observe changes in prescribing over time in other drug classes that the intervention did not address. Only 12.5% of eligible physicians fully engaged with this voluntary A&F initiative during the first 6 months of its availability. Interestingly, the physicians who engaged with the A&F initiative were already slightly less likely to prescribe antipsychotics at baseline, suggesting a latent interest in the topic. Our analysis identifies that certain characteristics were associated with physicians who voluntarily engaged. Physicians working in larger urban nursing homes and for whom nursing home residents represented a greater proportion of their practice seemed most likely to engage. This suggests relatively successful recruitment of higher-volume physicians. We also found that foreign medical graduates were less likely to sign up for and view the reports. Other studies examining physician characteristics associated with low-value care have also identified that foreign medical graduates may be more likely to over-test or over-treat [25]. This may reflect differences in social networks between early adopters and relative laggards [26] as it relates to engagement with data to inform practice. Since prescriber characteristics are associated with antipsychotic medication prescribing independent of resident and nursing home characteristics, an adaptable approach to implementation interventions that allows for recipient customization may be beneficial [27]. Prior research has shown that antipsychotic medication prescribing in Ontario nursing homes may be even more strongly associated with home-level characteristics than prescriber characteristics [2]. This, along with our finding of variable uptake for this provider-focused intervention, indicates a potential role for organizational-and systemlevel initiatives alongside provider interventions.

Implications
Prior research indicates that A&F can be effective, especially for prescribing [28], but the extent of effectiveness varies with the characteristics of the intervention [16]. Much research has focused of late on optimizing the design features of feedback [29]. Yet, regardless of how carefully designed the intervention is, feedback cannot be effective if the intended recipient does not engage. Our findings build upon prior work in Ontario indicating that many physicians do not actively engage in existing A&F initiatives [30][31][32][33][34]. Immediate clinical tasks may take priority, and many physicians, whether working in teams or independently, are struggling to keep up rather than looking for ways to get ahead [35]. It is possible that engagement will increase over time as the intervention matures, but it would appear that further research is needed to support both engagement with and action upon clinical performance data.
To address the barrier between signing up and viewing, Health Quality Ontario now sends the reports as email attachments (obviating the need to login to the passwordprotected website and manually download the report). In addition, since the time of this analysis, sign up rates have increased to over 400 physicians working in nursing homes (plus about 3000 physicians working in officebased primary care). This greater engagement over time reflects an important issue for this study-we purposefully analyzed the initial release of the report to understand early uptake and impact. Future research is needed to understand why some physicians were more likely to engage early in voluntary A&F interventions and how to leverage this information to increase the spread, scale, and impact of A&F and other implementation interventions.

Limitations
A number of additional caveats must also be highlighted in this study in terms of interpreting the effects on prescribing. First, although we adjusted statistically for measurable confounders, the non-experimental approach cannot be used to attribute causality with confidence; there may be additional confounders we could not capture. Those who voluntarily engaged in the intervention may be different from those who do not in ways that influence prescribing but cannot be captured using administrative data. These include the staffing models in the homes, the use of physical restraints or other techniques (whether appropriate or otherwise) to manage behavioral challenges, and the clinical rationale (whether appropriate or otherwise) for using antipsychotics. While the lack of effects seen in the tracer and balance outcomes supports conclusions regarding reductions in inappropriate antipsychotic prescribing, it is possible that other compensatory changes in prescribing occurred [36]. Second, the outcome, while objectively and reliably measured independently from the intervention, represents dispensing, not actual pill-taking. Indeed, all the measures used in this study that leveraged routinely collected administrative data were not created to answer the research question posed herein. Fortunately, the risk of measurement bias arising from this should be nondifferential across exposure groups. Third, we examined prescribing in three quarterly intervals (one pre-and two post-intervention quarters), using multivariable linear random effects regression with the individual resident nested within homes as the unit of analysis. An alternative approach utilizing additional pre-intervention measures in smaller time intervals (e.g., monthly) could have strengthened our ability to draw causal inferences, but would have required us to make additional model assumptions about the nature of the pre-intervention trend and type of intervention effect, as well as the type of correlation structures over time. Fourth, the data do not permit exploration of practice models that incorporate non-physicians, including nurse practitioners, as the prescriber. Likewise, the feedback was directed solely at physicians. In a team-based environment such as nursing homes, there may be a role for data that supports changes in processes for all team members, as appropriate. Finally, the methodological approach cannot explain why or how changes occurred, or whether initial changes in prescribing were sustained. Three to 6 months appears to be enough time to observe initial changes, but further research is needed to understand how the effects of this sort of intervention may vary over time (i.e., learning and decay effects).

Conclusion
In summary, we used population data and objective outcomes to pragmatically assess the early effects of a realworld initiative, finding that amongst those who engaged with the intervention, a statistically significant reduction was achieved. We explored the key implementation outcome of engagement with the intervention and identified variation in characteristics across those who did and did not engage. Just as drugs do not work in people who do not take them, A&F cannot work if recipients do not fully engage with their data. When it comes to A&F, the adage "if you build it, they will come" simply does not apply. It would appear that in a context where physicians are independent and autonomous contractors, facilitating engagement in quality improvement must be viewed as a long-term project.