The health and economic impact of implementation strategies for improving detection of hereditary cancer patients—protocol for an in-depth cost-effectiveness evaluation with microsimulation modelling

Abstract

Background

Patients with Lynch syndrome (an inherited cancer predisposition syndrome) remain largely underdiagnosed despite clinically and cost-effective testing strategies to detect patients. This is largely due to poor referral rates for high-risk patients for consideration of genetic testing. Targeted approaches to improve the implementation of guidelines and thus uptake rates of genetic testing require the use of limited and valuable healthcare resources. Decision makers must carefully balance the potential health impacts of implementation approaches against the associated costs, similar to when assessing the direct impact of health interventions. This protocol outlines the methods used to conduct an economic evaluation of different implementation approaches aimed at improving referral rates of high-risk patients, including estimating implementation approach costs.

Methods

A cluster randomised controlled trial (the Hide and Seek Project, HaSP) is underway to compare two different implementation approaches aimed at improving referral rates, and thus detection, of Lynch syndrome among colorectal cancer patients across eight Australian hospital networks. An in-depth process evaluation is being conducted alongside the trial and includes measures to collect comprehensive data on both implementation and intervention costs. These costs, in addition to HaSP outcome data, will be incorporated as inputs into an existing microsimulation model—Policy1-Lynch—to project the downstream economic and health impacts and determine the more cost-effective implementation approach from the Australian healthcare perspective.

Discussion

The ability to model the impact of different implementation approaches will enable the most efficient way of improving Lynch syndrome detection. The approach used in this study could also be applied to assess other implementation approaches aimed at increasing the uptake of cost-effective health interventions.

Trial registration

ANZCTR, ACTRN12618001072202. Registered on 27 June 2018.

Contributions to the literature

• We report a novel example by which alternative hospital-based implementation approaches can be integrated into a detailed health economic model to predict their downstream health and economic impacts, thereby determining cost-effectiveness.

• We provide a practical solution to capturing resource use associated with the implementation approaches and intervention strategies, with the latter potentially varying significantly by site.

• Integrating implementation costs into existing modelling platforms may be useful in guiding resource allocation to improve uptake of hospital-based health interventions and may be applicable to a range of other health conditions beyond Lynch syndrome.

Background

Economic evaluation is used to determine whether a health intervention is cost-effective, i.e. whether the improvement in health outcomes justifies the required healthcare resources. However, finding a cost-effective health intervention or issuing evidence-based best practice guidelines does not guarantee optimal uptake and implementation in health services. Additional resources may be required to enhance the uptake of cost-effective health interventions [1]. However, limited resources are available within the health system for such efforts [2]. Different implementation approaches can be used to identify the interventions and improve uptake, for example educational sessions or software modifications. In order for health system managers and policymakers to justify the allocation of resources for the implementation approach and subsequent interventions to improve uptake, there is a need not only to establish the clinical effectiveness of the implementation approach, but also the cost-effectiveness [1, 3]. However, the costs and effects of alternative implementation approaches are generally not considered in assessing the cost-effectiveness of a new health care intervention in standard Health Technology Assessment (HTA) processes.

A complete representation of costs is necessary to guide decisions about resource allocation within the health system. The true cost of implementation depends not only on the costs of the interventions used to improve uptake, but also on the implementation approach used and the service delivery setting through which it is deployed [4]. However, relatively few implementation studies to date have incorporated cost data in their reporting [1], of which most have focussed only on the subsequent intervention costs (i.e. without factoring of implementation costs) [4].

Lynch syndrome is an autosomal dominant hereditary cancer predisposition conferring an increased risk of colorectal, gynaecological and other cancer types [5]. Tumour-based testing of Lynch syndrome-associated cancers (through mismatch repair immunohistochemistry (dMMR) and/or microsatellite instability (MSI) testing) offers the ability to detect at-risk patients who warrant referral to specialist genetic services for germline genetic testing to establish a Lynch syndrome diagnosis [6]. Tumour microsatellite instability or loss of immunohistochemical expression of MMR proteins (without evidence of somatic inactivation as indicated by the presence of the BRAF V600E mutation or MLH1 promoter hypermethylation) indicates high probability of a pathogenic germline mutation in either of the four mismatch repair genes—MLH1, MSH2, MSH6 and PMS2—causing Lynch syndrome [7]. Diagnosing Lynch syndrome can have long-term health benefits both for the index patient and their at-risk relatives, with identified carriers having access to risk management strategies (such as colonoscopic surveillance, risk-reducing surgery and aspirin prophylaxis) proven to reduce cancer incidence and mortality [8,9,10,11].

For implementation approaches to be of good economic value, (1) the health intervention targeted for improved uptake must itself be both clinically and cost-effective and (2) the costs of the implementation approach must be justified by the extent of improvement on intervention uptake [3]. Studies have found that the recommended Lynch syndrome tumour testing and referral pathway (the direct health intervention being targeted in this study) is clinically and cost-effective [12, 13]. For example, Kang and colleagues found that when compared to no testing, the cost-effectiveness ratio of universal dMMR tumour testing strategies for Lynch syndrome ranged from $28,915 to $31,904 per life-year saved (LYS) from the Australian healthcare provider (Medicare) perspective, with the potential to prevent up to 80 colorectal cancer deaths in Australia per year [12]. In this study, the dMMR tumour testing strategies considered were either via immunohistochemistry or an MSI test, followed by a reflex somatic mutation test (either a BRAF V600E or MLH1 promoter hypermethylation test). Confirmed LS carriers underwent regular colonoscopic surveillance. Based on an indicative willingness-to-pay threshold of $30,000–$50,000 per LYS, this approach is likely to be considered cost-effective by decision makers [14].

Despite this, uptake of the Lynch syndrome tumour testing and referral pathway remains suboptimal [15,16,17]. An estimated 53% of Australian laboratories are yet to adopt a universal Lynch syndrome tumour testing strategy [18], and even when tumour testing is performed, genetic referral rates for those with abnormal results (indicating a high risk of Lynch syndrome) are poor. Two recent Australian studies demonstrated that a minority (34% and 26%, respectively) of colorectal cancer patients with abnormal tumour test results were referred for genetic counselling and testing [16, 17]. Similar findings have also been demonstrated in the international setting [19,20,21], highlighting a need for implementation interventions to improve genetic referral and Lynch syndrome diagnosis.

The Hide and Seek Project (HaSP) is a cluster randomised controlled trial (RCT) aimed at improving detection of Lynch syndrome among colorectal cancer patients across eight large Australian hospital networks [22]. Such efforts often require multisystem behaviour change among health professionals, and theoretical frameworks have been recommended in the implementation science literature to maximise opportunities for success [23,24,25]. However, there is little direct evidence as to whether this approach improves uptake over usual (non-theory informed) change processes, the costs of these approaches, and whether this translates to more cost-effective implementation. The HaSP trial compares two structured processes (the ‘implementation approach’) for identifying barriers to LS tumour testing and referral and designing and implementing targeted intervention strategies.

Improvements in Lynch syndrome referral rates are likely to result in many downstream health and economic consequences, for example via increased number of germline genetic tests ordered, increased health-service utilisation for screening and risk management, and ideally, prevention/early detection of Lynch syndrome-associated cancers. The authors and collaborators are involved in a body of work dedicated to projecting long-term outcomes for a number of cancer streams, including the use of a detailed modelling platform ‘Policy1’, which is an individual-based (microsimulation) discrete-event framework [12]. Policy1-Lynch is a health economic model platform simulating testing and surveillance strategies for Lynch syndrome, incorporating the natural history of Lynch syndrome-related cancers [12]. The HaSP trial is uniquely positioned to incorporate the costs of the two trial implementation approaches (theory-based versus non-theory based) into Policy1-Lynch to predict their downstream clinical and economic implications. The ability to incorporate implementation costs into such models has been demonstrated in the context of mass media campaigns aimed at improving adherence to population-based cancer screening programmes [26] but is yet to be applied for hospital-based behaviour change implementation interventions.

Using methods described in this protocol, this study will conduct a cost-effectiveness analysis of the implementation approaches being used in the HaSP trial to improve detection of Lynch syndrome among colorectal cancer patients. Specifically, we aim to address the following objectives:

• 1. Perform a costing study using a real-time process evaluation to document the resources associated with (a) the implementation approaches being tested in the HaSP trial, (b) the site-specific interventions developed, and (c) the subsequent implementation of these intervention strategies.

• 2. Incorporate HaSP costs and outcomes (changes in Lynch syndrome referral rates) for each intervention strategy and implementation approach into a microsimulation model Policy1—Lynch.

• 3. Simulate each of these intervention strategies and implementation approaches to determine, and compare, their cost-effectiveness. The cost-effectiveness of the implementation approach will be assessed in terms of the associated direct costs as well as the associated downstream health services costs (for example downstream colonoscopy surveillance costs) in relation to the downstream consequences for prevention of cancer in probands and their relatives.

Methods

The overall purpose of the economic evaluation will be to determine the cost-effectiveness (outcome) of a theory-based versus a non-theory-based implementation approach (comparator 1), and no implementation approach (comparator 2) to improving Lynch syndrome detection in high-risk patients (population). The analysis will combine clinical trial outcomes (changes in Lynch syndrome tumour testing and referral practices) and cost data. Measures for cost collection have been incorporated into an in-depth process evaluation conducted simultaneously alongside the HaSP trial [27]. Analysis will be performed in three stages: a HaSP cost analysis (stage 1), incorporation of HaSP costs into a microsimulation model (stage 2) and a cost-effectiveness evaluation (stage 3).

Summary of the HaSP trial and process evaluation

The HaSP trial [ACTRN12618001072202] is a cluster RCT comparing a theory-based implementation approach against a structured implementation approach (without the explicit use of theory) for improving Lynch syndrome referral practices for colorectal cancer patients and is currently underway across eight large (> 500 beds) Australian hospital networks [22]. Each site has been randomised by state to either a theory-based or a non-theory implementation approach, the primary outcome being changes in appropriate completion of the recommended Lynch syndrome tumour screening and genetic referral pathway within 2 months of colorectal cancer resection [22]. At each hospital network, a locally employed healthcare professional has been appointed at 0.2 full-time equivalent and trained as an ‘Implementation Lead’ to coordinate the implementation approach at their site. Implementation Leads are overseeing the following phases over a 2-year period: (1) baseline audit of Lynch syndrome referral rates, (2) formation of multidisciplinary ‘Implementation Teams’, (3) identification of target behaviours to achieve practice change, (4) identification of barriers to change, (5) generation of intervention strategies, (6) support of staff to implement intervention strategies, and (7) evaluation of intervention effectiveness using audit and process evaluation data. The theoretical components distinguishing the two trial arms take place in phases 4 and 5. Ongoing support is being provided by the research team (who have expertise in implementation science and behaviour change theory) via scheduled teleconferences prior to the rollout of each of the seven phases. Clinical data will be extracted pre- and post-implementation, the primary intervention outcome being the proportion of patients with risk-appropriate completion of the Lynch syndrome tumour testing and genetic referral pathway within 2 months of colorectal cancer resection.

In addition to determining the clinical effectiveness of the implementation approaches being tested in the HaSP trial (e.g. the approach resulting in greater improvement in Lynch syndrome referral), there is a further need to assess their comparative costs. A mixed-methods, theory-driven process evaluation is being undertaken in parallel to the HaSP trial in each of the hospital networks [27]. Guided by the UK Medical Research Council guidance statement on process evaluations [28], qualitative and quantitative data will be obtained from various sources throughout each of the HaSP phases (e.g. interviews of Implementation Leads and Lynch syndrome stakeholders, pre-post intervention implementation questionnaires, observation of multidisciplinary team meetings, fidelity checklists, and analysis of project logs) to understand the factors influencing the primary HaSP trial outcome. This process evaluation therefore also provides a means for documenting the costs and resources used for both the implementation approach and the intervention strategies developed. Aligning with new recommendations in the implementation science literature, findings from the economic evaluation will be interpreted alongside contextual information and stakeholder perspectives obtained from the qualitative process evaluation data [29]. Full details of the qualitative measures and analysis plan are available elsewhere [27]. Methods for collecting and evaluating cost data have been incorporated in the process evaluation and are summarised below and in Fig. 1. This protocol has been reported in line with the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist (see Supplementary File 1: CHEERS Checklist [30])

Fig. 1

HaSP flowchart with cost data collection

Measures for cost data collection

Implementation Lead project log

Implementation Leads will complete a detailed Project Log prospectively for each HaSP phase (see Additional file 2 for example) to document the time and resources involved for each implementation approach. Implementation Leads have been provided with training to understand the purpose of the Project Log, including the type of information to be recorded and the level of detail required. As a guide, phase-specific example Project Logs have been provided to Implementation Leads to complete throughout the trial. Using the Project Log template, each Implementation Lead will be asked to document the steps taken to complete each task, and the time and resources involved. For meetings and focus-groups held for HaSP trial purposes (see Fig. 1), the number and roles of hospital staff attendees will also be recorded. At the completion of each phase, Implementation Leads will send the research team the completed Project Log for analysis.

HaSP team cost log

The HaSP research team (based at Cancer Council NSW) will also retrospectively document the time and resources used to train and support the Implementation Leads in overseeing the trial at each site. For each phase and hospital site, a detailed breakdown of planned tasks will be built into a Microsoft Excel spreadsheet. A member of the HaSP team (AM) will be responsible for collecting staff time and resources for each task, to collate costs for each Phase. This allows a streamlined process for costs to be assigned to specific HaSP tasks, and categorised and modified across all study sites (see Additional file 3 for example).

Fidelity checklists

Site-specific intervention strategies aimed at improving Lynch syndrome referral rates will be developed and implemented over a 6-month period (Fig. 1). However, the costs associated with the implementation of the interventions throughout this period will be dependent on staff uptake, and whether the strategies are delivered as intended (intervention fidelity). Site-specific fidelity checklists will be developed to monitor the ongoing implementation of each intervention strategy, and will be completed by the Implementation Lead and a second Implementation Team member every three weeks throughout this period. Observation of meetings and focus-groups held for HaSP trial purposes (Fig. 1) will also be undertaken to assess adherence to, and uptake of, the prescribed implementation approach. These fidelity assessments will be used in combination with the Implementation Lead Project Log for later sensitivity analyses.

Cost assignment

Intervention costs

The intervention strategies designed by the Implementation Teams may vary significantly across sites, depending on the target behaviours and barriers identified. A co-design strategy will be used, with the research team working closely with the Implementation Teams in the development and implementation of the intervention strategies. Accordingly, resources required for each intervention strategy will also vary depending on start-up and ongoing costs (including recurrent and capital items) and the organisational level(s) and department(s) involved [31]. For example, one site may implement a once-off, educational intervention to improve Lynch-syndrome knowledge among surgical and oncology staff in an effort to improve genetic referral practices. This would involve costing of the initial start-up resources needed to develop and deliver the educational package, as well as staff time for attendance (see ‘Unit Costs’ below), after which there would be no anticipated ongoing costs. In contrast, another site may opt to redesign their pathology ordering system, which may require capital costs for software modifications, in addition to ongoing costs associated with an increased number of immunohistochemistry tests ordered and pathologist time.

Individualised costing plans will therefore be developed for each site and planned intervention strategy. This will be done in consultation with a health economist (BP) during the intervention design stage. Members of the research team will then work closely with Implementation Leads and (where relevant) hospital staff members to ensure unit costs are collected according to plan. These costs will also be documented in the Implementation Lead Project Log and HaSP Team Cost Log.

Unit costs

Unit costs applied to the time of Implementation Leads and hospital staff (derived from Implementation Lead Project Logs) will be determined by their role and the average salary per hour [32]. Invoices will be retained for consumables (e.g. travel costs, stationary costs and printing costs) and recorded in the HaSP Team Cost Log. Where relevant, unit costs for pathology tests (e.g. mismatch repair immunohistochemistry, BRAF V600E immunohistochemistry) will be based on Medicare Benefits Schedule (MBS) item numbers [33], or via local laboratory price lists (for tests without Medicare Benefits Schedule item numbers). Unit costs associated with the potential downstream clinical consequences (e.g. genetic counselling consultations, germline test ordering) of the intervention strategies and implementation approaches employed have been estimated a priori through the microsimulation model Policy1-Lynch [12]. Unit costs will be dated to allow later adjustments to the year of reported costs (where necessary).

Analysis

HaSP-related cost analysis (stage 1 analysis)

In the first stage of analysis, a costing study will be performed for each site to determine the resource use associated with each implementation approach and intervention strategy. Total costs will be estimated using cost data collated from the Implementation Lead Project Logs and HaSP cost spreadsheet.

There may be additional research and development-related expenses which would not be applicable if translated to a ‘real-world’ setting (e.g. study-specific training materials and implementation resources, ongoing research team support to ensure fidelity to the prescribed implementation approach). Consequently, it is necessary to distinguish the costs assigned for research and development purposes to assess whether the implementation intervention would be cost-effective in the ‘real-world’ clinical practice setting [34]. Accordingly, costs will be classified according to whether the costs are fixed or ongoing, universal or site-specific and whether they are incurred primarily for research purposes (e.g. processes that would apply only in the research implementation setting) or implementation purposes (e.g. processes that would apply in the real-world implementation setting). Mean costs and standard errors by each of these categories will then be estimated for each of the implementation approaches. Costs will be analysed from the Australian health system perspective, according to guidelines, and reported in 2020 Australian dollars [35].

Incorporating HaSP costs into a microsimulation model ‘Policy1-Lynch’ (stage 2 analysis)

In the second stage of analysis HaSP-related costs (as determined from stage 1) and outcome data will be incorporated into Policy1-Lynch. Policy1-Lynch is a comprehensive health economic model platform to simulate pathways for testing (both for proband and at-risk relatives), diagnosis, surveillance and prophylaxis for Lynch syndrome and has several core components, including a model of cancer-specific natural history. It requires information (i.e. parameter inputs) such as implementation costs, germline genetic testing uptake rate, and adherence rate for colonoscopic surveillance to produce health and economic outcomes. Policy1-Lynch is a microsimulation model and the current version simulates colorectal cancer in Lynch syndrome carriers and non-carriers throughout their lifetimes. Colorectal cancer development in Lynch syndrome carriers is modelled via application of cumulative colorectal cancer risks with and without colonoscopic surveillance [10, 36]. Colorectal cancer incidence rates for non-Lynch syndrome individuals in 2021 (year at study completion) were based on population-based sex- and age-specific colorectal cancer incidence observed in Australia in 2015 (the latest available data) [37]. Individuals were assumed to be able to develop up to two colorectal cancers in a lifetime (i.e. up to one metachronous colorectal cancer). The model and underlying data sources have been previously described in detail [12].

HaSP-specific information will be used as model parameter inputs to Policy1-Lynch, including: the range of costs associated with each intervention strategy (e.g. educational workshops, referral reminder systems) and implementation approach (input 1), changes in Lynch syndrome referral rates (input 2), and other measurable clinical practice changes resulting from HaSP (e.g. increases in pathology and germline genetic testing rates) (input 3).

Cost-effectiveness analysis of implementation strategies (stage 3 analysis)

In the third stage of analysis, each HaSP implementation approach and intervention strategy will be simulated in Policy1-Lynch to predict the downstream health and economic outcomes. Figure 2 provides an example of how implementation costs can be integrated into Policy1-Lynch based on current standard practice. These simulations will be used to determine (a) the most cost-effective implementation approach and (b) the most cost-effective intervention strategies. Whilst the primary aim of the economic evaluation is to determine the more cost-effective implementation approach, the model will also allow us to compare either approach to an additional comparator: no implementation approach (e.g. ‘practice as usual’; comparator 2).

Fig. 2

Policy1-Lynch model with integrated implementation costs. (a) ‘Not LS’ includes normal IHC/MSI results (preserved immunohistochemical MMR expression) as well as abnormal IHC/MSI results (loss of immunohistochemical MMR expression) with BRAF V600E mutation or with MLH1 promoter hypermethylation. (b) High risk of LS includes abnormal IHC/MSI results without BRAF V600E mutation or without MLH1 promoter hypermethylation. (c) Timing of referral to genetic counselling varies by states. For simplicity, we assume all patients at high risk of LS are referred to genetic counselling, then offered diagnostic germline genetic testing. Therefore, CRC patients at high risk of having LS who are not compliant to genetic counselling do not take up diagnostic genetic testing, and their relatives, who may or may not be LS carriers, do not receive predictive testing and LS surveillance. (d) We do not explicitly model standard care after cancer treatment; however, stage-specific cancer treatment costs include the cost of initial treatment, the cost of follow-up appointments, blood tests and imaging, treatment for relapsed disease and palliative care. BRAF, B-Raf protooncogene; CRC, colorectal cancer; dMMR, mismatch repair deficiency; F/U, follow-up; IHC, immunohistochemistry; LS, Lynch syndrome; MLH1, mutL homolog 1 gene; MSI, microsatellite instability

The time horizon of the analysis will be lifetime. A discount of 5% will be applied for each 12 months after study rollout [35]. The incremental cost-effectiveness ratio (ICER) will be estimated to compare the two HaSP implementation approaches (theory-based vs non-theory based), with the main health outcome expressed as life-years saved. Other health outcomes, such as cost per additional patient tested and per death avoided, will also be reported. If the ICER is below a range of A$30,000 to A$50,000 per life year saved, it will likely be considered cost-effective by decision makers [38].

A range of scenario analyses and sensitivity analyses will be conducted to investigate the effects of key assumptions and parameters on the cost-effectiveness of each implementation strategy. These will include, but are not limited to, removal of intervention costs, implementation costs, research costs and implementation outcomes (e.g. fidelity and adoption) as well as a range of referral rates and genetic testing uptake rates. A range of ICERs will be evaluated as assumptions around both the implementation approach and comparator effect change. Evaluating a ‘no implementation’ scenario will allow us to account for potential general improvement in referral rates over time unrelated to the HaSP implementation approach.

Discussion

To our knowledge, this will be the first study to incorporate alternative hospital-based implementation approaches into a detailed health economic model to predict their downstream health and economic impacts. This study will also be the first to evaluate the comparative resources associated with a theory-based implementation approach, and whether this translates to greater cost-effectiveness than a non-theory based implementation approach. In doing so, we address a number of key research agenda items put forward by Dopp and colleagues [29] to advance traditional economic evaluation methods into the implementation science space.

Whilst most health system economic evaluations focus only on intervention costs, the HaSP trial process evaluation will provide comprehensive cost data encompassing implementation training, through to intervention design and development, and the eventual implementation of intervention strategies across a range of service delivery settings. The ability to then incorporate these costs into Policy1-Lynch will allow us to predict the potential impact of such efforts across a range of hypothetical scenarios and settings. Interpreting these findings alongside qualitative HaSP process evaluation data will provide a contextually grounded picture of the costs and impacts of implementation [29].

If one of the HaSP implementation approaches proves to be both clinically and cost-effective, transferring the approaches from the research setting to a clinical setting will require considerations for scalability [39]. Detailed collection of resource use and conducting multiple sensitivity and scenario analyses may offer the ability to identify areas to reduce the costs associated with the implementation approaches. Adjustments can then be made to the implementation cost inputs for Policy1-Lynch to predict their impact at a greater scale.

Such work is timely given the changing landscape of genetic and genomic testing, particularly in light of the recent introduction of MBS-funded germline genetic tests for cancer patients with suspected Lynch syndrome (on the basis of tumour screening results) [40]. Whilst clinical cost projections provided justification for the new funding scheme, the costs associated with formally implementing the new testing strategy are yet to be assessed. Enhancing the uptake of an intervention requires resources; however, there are limited resources available within the health system that could be used for other purposes.

This study protocol is not without limitations. Firstly, the small number of sites may limit the generalisability of the mean costs of the two implementation approaches. However, we have sought to overcome this by recruiting large hospital networks, across a number of states, to ensure outcomes are representative in an Australian setting. Furthermore, Policy1-Lynch currently focusses on life-year (rather than quality-adjusted life year) outcomes as further research is needed to estimate health state utilities associated with Lynch syndrome [12]. It is not clear whether the various implementation approaches may have differing effects on downstream quality of life consequences for patients. These limitations will need to be factored into consideration when using the model for decision-making purposes.

Nonetheless, the current study provides a novel example by which the costs of various implementation approaches can be modelled to determine the most efficient use of health resources for enhancing uptake of clinical practice guidelines, within and beyond the context of Lynch syndrome.

Appendix

Table 1 : Implementation lead project log template

Table 2 : Hide and Seek Project – example log for Phase 3: Identify target behaviours for change