Prioritizing research needs and opportunities at the intersection of implementation science and engagement science

Abstract

Background

There has been increased attention to the need for, and the positive impact of, engaged or participatory science in recent years. Implementation scientists have an opportunity to leverage and contribute to engagement science (ES) through the systematic integration of engagement into implementation science (IS). The purpose of this study was to gather information from researchers and others to develop a prioritized list of research needs and opportunities at the intersection of IS and ES.

Methods

We conducted three Zoom-based focus groups with 20 researchers to generate a list of unmet needs, barriers, and to describe normative themes about use of ES and IS. Then a panel of nine experts in IS and/or engagement ranked the needs and barriers using a survey and met via a Zoom meeting to discuss and generate research opportunities and questions, with reference to the focus group outputs.

Results

Respondents and experts concurred on the importance of engagement in IS. Focus group participants reported 28 needs and barriers under the themes of 1) need for best practice guidance related to engagement processes and outcomes and 2) structural barriers to integrating ES in IS. The expert panel prioritized six structural barriers and four barriers related to generating best practice guidance, with corresponding recommendations on research opportunities. Example research opportunities related to engagement processes included: define “successful” engagement in IS contexts; adapt engagement tools and best practices from other disciplines into IS. Example research opportunities related to outcomes included: assess the impact of engagement on IS outcomes; examine engagement practices that lead to optimal engaged research. Example research opportunities related to structural barriers included: leverage research evidence to create structural changes needed to expand support for engaged IS; examine factors that influence institutional buy-in of engagement in IS.

Conclusions

Research needs exist that relate to engagement processes, outcomes, and structural barriers, even for scientists who value engaged research. Expert panelists recommended sequential and reinforcing research opportunities that implementation and engagement scientists can tackle together to advance both fields and health equity. Future work should assess insights from broader invested parties, particularly patients and community members.

Contributions to the literature

• This study responds to calls in the literature to increase community and partner engagement/participation by outlining research-related needs (e.g., recommendations, training, and structural changes) and scientific opportunities for integrating the study of engagement into dissemination and implementation research.

• These findings contribute to a gap by providing specific recommendations for how implementation scientists and researchers with engagement expertise can collaborate to advance critical questions relevant to both fields.

• These findings also speak to funders and research administrators, highlighting the unresolved barriers and structural changes needed to enable more engaged or participatory research, especially in partnership with underrepresented communities experiencing disparities.

Background

Implementation science (IS) is the study of methods to promote the adoption and integration of evidence-based practices, interventions, and policies into routine healthcare and public health settings to improve the impact on population health [1]. As a field, IS benefits from engagement with healthcare and public health decision makers as well as real-world implementers and end users of evidence-based practices, interventions, and policies [2]. Recent evidence suggests that implementation scientists frequently conduct engaged or participatory research, though the level or depth of engagement varies [3].

IS professionals have called for increased focus on integrating engagement in IS in conjunction with an increasing focus on health equity and health disparities [4,5,6]. A 2023 scoping review by Gustafson et al. identified twelve IS theories, models, and frameworks (TMFs) with an equity focus [7]. Engagement of communities experiencing disparities was emphasized in most of them, particularly during implementation planning phases. Likewise, Wilkins et al. (2023) recently outlined how community-engaged research is essential to addressing health inequities [8]. Research focused on equity and disparities demands engagement with populations who are often marginalized and under resourced, to foster trust and support community empowerment and self-determination. Ramanadhan et al. (2023) provide a review of participatory IS to advance health equity that includes a rationale for this type of research, a framework for integrating participatory approaches into IS TMFs, as well as important considerations for how to do this work well [9].

Outside of IS, there is a long, rich history of engaged research that includes various models and approaches for engaging different groups in the pursuit of optimal opportunities for individuals or populations to attain the best health possible (i.e., health equity). Perhaps the most well-known because of its long history is Community Based Participatory Research, or CBPR. CBPR, as defined by the Kellogg Foundation, “equitably involves all partners … with a research topic of importance to the community with the aim of combining knowledge and action for social change to improve community health and eliminate health disparities” [10]. The Patient-Centered Outcomes Research Institute (PCORI) is a major funder of engaged healthcare research, including CBPR, and their model requires “meaningful involvement of patients, caregivers, clinicians, and other healthcare stakeholders throughout the entire research process—from planning the study, to conducting the study, and disseminating study results” [11]. However, several consistent barriers and challenges to conducting this type of highly engaged research have been documented, including the increased level of effort, time, and expertise in engagement methods required [9]. Further, not all IS studies require the same level of participation or engagement from the community or other partners, depending on the focus of the research. Therefore, other approaches and strategies may be most appropriate at intermediate (cooperation, collaboration) and minimal (outreach and education) levels of engagement [12].

There is a large body of evidence around the benefits of community engagement. Ramanadhan et al. 2023 highlight both instrumental (e.g., more relevant research, better understanding of local context) and normative (e.g., enhanced justice, inclusion, and equity) motivations for engaged research [9]. A 2020 meta-review of community-engaged scholarship by Ortiz et al. examined 100 reviews of research, 55 of which reported on partnership or health outcomes. They concluded that evidence largely supports community-engaged research contributing to improved health and health equity, as outlined in the CBPR conceptual model [13]. In addition, a qualitative analysis of the impact of engagement in 58 PCORI-funded studies found that researchers and their partners alike viewed engagement as worthwhile and noted it influenced the design and execution of the studies and was integral to the success of the research [14]. Combined with IS, engaged research approaches have great potential to move closer to health equity [15].

Given the prevalence of engagement activities in IS, it could be an opportune marriage to incorporate unanswered engagement-related scientific questions and measures into IS studies. Distinct from engaged research that uses various techniques for enabling input and participation of non-researchers, engagement science (ES) examines the methods and outcomes of engagement to build an evidence base for the value and quality of engagement [16,17,18]. As such, the 15th Annual Conference on the Science of Dissemination and Implementation included a forum titled, “Research Opportunities at the Nexus of Engagement Science and Implementation Science” [19]. Twenty-four attendees worked in small groups to identify important and opportunistic engagement-related scientific questions that could be answered through IS. Building upon this initial forum and brainstorming activity, the present study reports on subsequent data collection and activities to develop a prioritized list of scientific needs and research opportunities for implementation scientists and funding agencies to consider.

Methods

We report on a series of focus groups about use of ES and IS, followed by discussion with expert panelists [20]. The purpose of the exploratory focus groups was to generate a diverse list of unmet needs, barriers, and to describe normative themes from the perspectives and experience of a multi-discipline group of participants. The purpose of the expert panel was to deliberate with the study team to structure and organize research opportunities and priorities for integrating ES and IS, using the needs and barriers generated from the focus groups as a starting point. This multi-step process was adapted based on several previous efforts to generate research priorities in other fields [21,22,23].

Focus groups

Participants

We applied purposive sampling techniques to recruit focus group participants from a list of 55 ES and IS researchers using published literature and professional contacts [24]. Targeted participants had a mix of expertise in ES and/or IS, and a range in number of years of experience. We emailed a prescreen survey to the list of prospective participants and ES and IS professional listservs to identify interested individuals and let them indicate their expertise.

Procedures

In advance of the focus groups, participants were provided a series of suggested background readings and definitions related to ES and IS. Three Zoom-based focus groups were conducted in August 2023 and lasted about 90 min, with five to nine participants each. Focus groups were facilitated by the senior investigator (HP) with another investigator (ER) using a structured template to capture notes [25]. The discussion prompts followed the topic areas described in Table 1. Participants were asked to provide input on barriers and needs related to integrating ES into IS. Participants were informed that an expert panel would subsequently convene to aggregate research priorities informed by the focus group data.

Table 1 Focus group topic areas

Analysis

Initial focus group analysis was conducted by SNH using a rapid analysis approach to summarize the comments into specific themes and subthemes [26]. Next, using consensus-based revisions involving HP, SJ, SNH, and ER, we converted these themes and sub-themes into a list of 28 specific needs and barriers.

Expert panel

Panelists and process

HP and AV selected and invited 14 individuals with notable scientific leadership and publications in ES and/or IS to serve on the expert panel, with input from the study team. The expert panel provided their feedback in two modes: completing a ranking activity and participating in one of two 90-min Zoom-based meetings. Participants ranked 100 randomly generated pairs of needs and barriers from the list of 28 identified by the focus groups, using an online-asynchronous free stack ranking tool OpinionX (https://www.opinionx.co/) (Fig. 1). We used the resulting item rankings to initiate the expert panel discussions. HP moderated the discussions, and ER and SNH took notes. The discussion began by HP requesting the experts generate research opportunities associated with the top ten ranked needs and barriers, though the discussion was not strictly limited to that list.

Fig. 1

Ranking activity example

Analysis

SNH used a rapid analysis approach to sort and refine the expert-generated research opportunities and map them to the focus-group generated barriers and needs [26]. Some barriers generated multiple research opportunities and a subset of research questions spanned multiple barriers. The resulting matrix was reviewed by SJ, who provided additional IS context, and then by SNH and ER for further refinement and interpretation. We used quotes (which came from either verbal or chat communication during the expert panels) to further contextualize each research opportunity and identify associated research questions. A summary document was emailed to panelists for member checking, and minor further refinements were made to the list of recommendations and opportunities based on feedback received.

Results

Focus groups

Focus group participants

The twenty focus group participants resided in the United States (n = 18), Nigeria (n = 1), or Norway (n = 1) and indicated they had expertise in either IS (n = 10) or both ES and IS (n = 10). Most participants (n = 11) identified their role as “professor.” Other roles included researchers (n = 4), project/program directors (n = 2), and post-doctoral student (n = 1).

Engagement value and approaches

Participants discussed the value of engaging with community members, patients, and other impacted parties to develop sustainable solutions to meaningful problems. Focus group participants spoke about engagement as a being “a moral imperative if using taxpayer dollars” with improved likelihood of “achiev[ing] the best performance when we engage communities.” Moreover, researchers felt that engagement was necessary for fostering trust and reciprocity. One researcher stated:

“Engagement is about trying to figure out how we stop just pushing our results out and build the relationships so that we’re answering the questions that people want… at its best, engagement is about making sure that the product we’re creating is the product that the community needs.”

Participants reflected on a range of engagement approaches (e.g., consultants versus project co-leads), the selection of which leads to differences in the impact of the engagement, the methods used, and the required resources. In addition, respondents described the different barriers and opportunities associated with the short-term compared to long-term engagement of partners (which builds strong relationships, capacity, and trust but requires investigator commitment to nurturing relationships between studies). Overall, our respondents viewed engagement as resource-intensive and requiring extra commitment and investment from the research team, yet they emphasized that the benefits of integrating ES into IS exceed the resources put in.

Needs and barriers related to integration of ES and IS

Focus group respondents generated a broad set of needs and barriers at the intersection of ES and IS categorized under the themes and sub-themes shown in Table 2 and discussed below.

Table 2 Needs and barriers related to integration of ES and IS by theme

Need for best practice guidance related to engagement processes and outcomes

A total of 15 barriers were categorized under this theme. These items reflected the need for consensus or evidence-based guidance on conducting engagement and measuring its impact. Most of the barriers reflect the need for practical “how to” guidance related to initiating, maintaining, and sustaining engagement. Examples include clearly defined engagement goals and processes, which engagement partners should be included, best practices for achieving shared goals across different types of engagement partners, engagement strategies to use at different stages of implementation and in different settings, and appropriate compensation for engagement partners. For example, when prompted to describe needs around processes, one focus group participant responded: “How to listen first to hear community needs—how to create space for communities to share their perceived needs and priorities.”

Respondents also reflected on the importance of consensus regarding meaningful ES outcome measures and integrated ES and IS TMFs. When describing how to measure impact of engagement, one focus group participant said, “We often rely on frequency metrics like number of activities held, but it can be misleading if quality of those engagements is lacking.” Another participant asked, “Who defines what success looks like?” Focus group participants noted the utility of an integrated model to support ES process evaluation and assessment of the impact of integrating engagement on implementation outcomes. As stated by one focus group participant, it is “not one size fits all. One research opportunity is to document and understand how engagement is done in different settings for different groups.” Two sub-themes consisted of barriers at the investigator and partner level, which all related to the need for guidance or capacity building on ES processes and navigating structural barriers.

Investigator-level Barriers

Respondents reported three investigator-level barriers to ES in IS. Two items relate to relationship building—one being the willingness and ability of the research team to develop and maintain relationships with engagement partners, and the other being the inherent effort and vulnerability stemming from the interpersonal relationships required to make engagement successful. Respondents described the importance of relationships among investigators and engagement partners to the establishment and maintenance of engagement. Investigators noted a need for guidance on facilitating engagement intensity and sustainability. Participants voiced particular concern about the maintenance of community-based projects. One focus group participant stated:

“I think about this in the community…I worry sometimes that some of my projects are one person away from basically just going out the window because they rely so much on these individual relationships. So how do you move past that? And I don’t know the answer, but I see that as a problem.”

A related concept was the need for guidance on activities that are effective at sustaining engagement with partners. Some respondents reflected on the professional and personal cost of using engaged research approaches, which may require more of investigators (e.g., work outside of standard hours, extra travel) while not being highly valued or recognized by some institutions. However, as noted above, many respondents also reported highly valued personal and professional benefits of conducting ES in IS.

Engagement Partner-level Barriers

Focus group members identified three engagement partner-level barriers. One of these barriers is mistrust of researchers and institutions, particularly among under-represented and marginalized communities. Respondents reported an imperative to engage with under-represented and distrustful communities, but also a need for guidance on how to build collaboration and trustworthiness across diverse research and engagement teams. The other barriers reflected the arcane vocabulary used in medicine and science and the unfamiliar and technical aspects of research and grant proposals. Respondents discussed the need for plain language materials that are accessible to partners so that all parties can provide meaningful input. Focus group members also described the need to familiarize engagement partners with the grant proposal and review process, the research process, and the structures and limitations that are imposed on IS by funding agreements, protocols, resource limitations, and regulations. Related to the critical need to translate technical information, a focus group participant described the parallel development of the IS approach and a guide for engagement partners:

“You know, for a community-based group…we kind of miss a gap in that it’s too conceptual. So even in some of the work that we’ve been doing with [institution name] we kind of think not only about an implementation blueprint, but then how can we create an adaptation guide where it’s very layperson friendly?”

Structural barriers to integrating ES in IS

The focus groups identified 13 structural barriers. Overall, respondents requested that institutions and funders remove roadblocks to successfully integrating ES in IS as well as reduce barriers to entry for engagement partners and investigators who are new to engagement. Under this domain were two subthemes.

Barriers generated by the research institution

Respondents described challenges associated with conducting engagement activities within a non-supportive or non-experienced institution and/or an environment where engagement is not incentivized. These challenges were described as procedural (e.g., institutional reticence to appropriately compensate engagement partners) and related to professional incentives (e.g., promotion). Institutional support for training was also needed to better prepare investigators; as one participant described, the need for ES “intentionally being part of graduate and post doc training.”

Barriers generated by the funding institution

Respondents reported several procedural or instrumental barriers generated by funding institutions. For example, traditional grant timelines and budgets do not account for the specific needs of engaged research, which is more time- and resource-intensive. Respondents also described ways in which typical grant processes and life cycle negatively impacted some aspects of investigator and engagement partner relationships, particularly with partners who are less familiar with research. Examples include attempting to maintain relationships once funding ends (in anticipation of future engaged research) and that long grant review timelines and low chance of funding was related to challenges with establishing community partnerships whose availability may shift once a study is funded or who may be less inclined enter into new partnerships later if a study is not funded initially. One focus group participant stated:

“How can we describe some of our wonky funding systems in academia with our engaged partners? Like I can work months with a partner to write a grant. We want to do this work, we get everyone excited and then six months later, I’m like, ‘well, sorry, we didn’t get funding.’”

While focus group participants agreed that structural barriers de-incentivized future engaged IS, they also agreed that this category was only minimally amenable to meaningful research questions.

Expert panel

Expert panelists

Of the 14 invited, 9 accepted the invitation to serve on the expert panel. Expert panel participants had expertise in ES and/or IS and are listed in Table 3.

Table 3 Expert panelists

Ranking of focus-group generated needs and barriers

The expert panel ranked the 28 needs and barriers in order of importance and the top 10 were used to initiate a discussion of research opportunities (Table 4). Six of the top 10 were structural barriers and the remaining 4 were needs related to generating best practice guidance for engagement processes and outcomes.

Table 4 Top ten needs and barriers as ranked by expert panel

Expert panelists’ discourse about research opportunities and priorities

Table 5 presents the expert panel recommendations on research opportunities important for increasing ES in IS organized around three areas, which stemmed from the focus group themes: (1) engagement processes, (2) engagement outcomes, and (3) structural barriers.

Table 5 Expert panel recommendations and research opportunities

Expert panelists agreed that there are needs and opportunities related to defining goals, processes, and outcomes of engagement and providing best practice guidance for conducting engaged IS. They also concurred that engagement must be suited to the context and objectives—there is not a one-size-fits-all approach. Panelists emphasized that data are needed to advance the field from practice-based guidance to evidence-based, and to demonstrate the impact of engagement ‘done well’ on implementation outcomes. Such evidence is needed to inform change and support adoption at the structural level. To be successful in achieving research opportunities, experts reflected on the importance of institutional support; employing committed multidisciplinary teams that include ES and IS experts; and meaningfully integrating invested and impacted parties (e.g., patients, community members, minoritized groups, clinicians and other ‘on the ground’ implementers) into the process, including in leadership roles.

Once engagement processes and outcomes are defined, expert panelists agreed that is critical to assess the impact of engagement on implementation outcomes. They also outlined several specific opportunities related to studying engagement processes and outcomes within IS studies. To facilitate this research, panelists endorsed using or adapting existing ES tools and best practices from other fields for IS studies, including a need for integrating ES into existing IS TMFs. These would need to be flexible enough to apply to a variety of settings and populations. Emphasis was placed on the need for integration rather than the creation of new TMFs, as one panelist stated, “We don’t want a new model. Just help us figure out how to integrate within models we’re already familiar with and whether that’s an engagement or an implementation model.” The desired updated TMFs were framed as “causal”, and could be used to determine how, why, when, and where to engage for the purpose of improving implementation. Panelists acknowledged the complexity of developing causal TMFs related to engagement, but agreed that it is a necessary step for providing intentional guidance in this area: “this to not only thinking through the goals and processes, but you know, developing an explicit causal model, what are the precise causal mechanisms and effects of engagement on the desired outcomes once we’ve identified those desired outcomes.”

Panelists agreed that structural changes are needed to make engagement more feasible, for engagement to be prioritized, and for engagement activities to be sustainable. They supported the idea that evidence generated from process and outcomes research could be used to reinforce the value of engagement and build the will for and guide necessary structural changes. Nevertheless, panelists agreed that structural level changes would be challenging but still needed to be addressed to maximize the impact of engagement-focused research.

Finally, expert panelists noted the importance of integrating additional viewpoints, particularly focused on patient and community perspectives, into future efforts to refine needs and research priorities. Panelists called out specific needs associated with engagement of minoritized populations that may require an intentional focus in IS research, engagement, and associated priority setting.

Discussion

Focus group participants and expert panelists echoed critical points found throughout the engagement literature about the positive value of, but challenges to, engaging partners and communities in research broadly and in IS in particular. A recent analysis of National Institutes of Health (NIH) funded IS grants found that 87% involved some degree of community or partner engagement [3]. However, most (60%) were categorized as outreach or consultation, the lowest levels on a continuum [12] of the extent to which a community can be involved in research [3]. This might stem from limited formal training and inexperience with engagement, which leads to a lack of sophistication in describing the approach, or a hesitancy to go deeper in the engagement with partners due to noted barriers. As evidenced in our study, there is clearly a desire to conduct engaged IS but needs and barriers remain in understanding what is being done currently and how to improve it. Six of the top ten needs and barriers ranked by the expert panel, not unique to IS, are structural barriers erected by funding agencies (e.g., grant timelines, budget limitations) and academic institutions (e.g., contracting and compensating partners, incentivizing engaged research). Another precursor need to enable integration of meaningful engagement in IS is evidence-based guidance on engagement processes and outcomes. Implementation scientists, in partnership with engagement scientists, can help generate this guidance by investigating the questions outlined by the panelists.

Of critical importance to advancing this work is the integration of engagement into IS TMFs and incorporation of engagement outcomes measures in IS studies. A 2021 review by Pinto et al. reported that only 20% of IS TMFs reviewed included the 5 community engagement constructs assessed and that few TMFs with community-related constructs has been introduced since 2009 [27]. This aligns with the panel recommendation that engagement and implementation scientists collaborate to determine which engagement TMFs are most useful for different IS needs or objectives and integrate engagement constructs into IS TMFs.

Despite the plethora of evidence of the importance of engaged or participatory research, some still diminish the largely qualitative evidence and question the impact of engagement [28]. Indeed, Ortiz and colleagues (2020) note in their review of reviews that synthesizing the community-engaged research literature was challenging due to the largely qualitative nature of research methods and reports. They encourage future research on the development of evaluation tools for community-engaged research and on tailored models or approaches for specific populations experiencing health inequities [13]. Similarly, expert panelists here recommended defining meaningful outcomes to measure the impacts or outcomes of engagement, assessing the impact of engagement on implementation outcomes, and they specified multiple specific research questions about engagement outcomes to integrate into IS studies. To facilitate a head start for investigators, the National Academy of Medicine’s Organizing Committee for Assessing Meaningful Community Engagement in Health and Health Care has outlined a conceptual model on the dynamic relationship between community engagement and outcomes and has begun to curate instruments and questions used in different contexts and communities to assess engagement [29, 30].

After years of funding research that required engagement with patients and others involved in healthcare, PCORI took a step toward addressing remaining critiques and skepticism about the value and outcomes of engagement with their Science of Engagement funding opportunity. First issued in 2022, this opportunity funds studies that build an evidence base on engagement in health research. The science of engagement is an emerging area of inquiry that moves beyond simply conducting engaged research to examine the engagement itself. PCORI’s funding opportunity focuses on two areas, with initial funding priority given to the first: 1) development and validation of reliable quantitative measures for engagement structure or context, processes, and outcomes and 2) developing and testing engagement techniques to generate evidence on the most effective engagement approaches, particularly for underrepresented populations [18, 31]. While this is promising in addressing some of the recommendations of the expert panel, other funders could also contribute to more rapidly moving the field forward by attending to recommendations on research opportunities related to the structural barriers outlined.

Many of the research opportunities presented here are sequential and reinforcing (Fig. 2). Agreement about defining successful engagement processes and outcomes is a necessary initial step prior to conducting engaged IS that can generate evidence related to engagement effectiveness and the impact of engagement on implementation outcomes, which is needed to offer evidence-based guidance on engagement processes. This evidence in turn may be helpful to efforts to garner support for needed structural changes at research and funding institutions, which would enhance the research context in favor of supporting engaged research.

Fig. 2

Sequential and reinforcing nature of ES in IS

If achieved, structural changes would facilitate conducting more IS with ES integrated, thereby advancing both the sciences of implementation and engagement for improved population health outcomes and health equity. As an example, NIH’s Community Engagement Alliance (CEAL) has developed the CEAL Consultative Resource as a channel for sharing best practices to NIH-funded research teams wanting to apply principles of community-engaged approaches to address health disparities and ensure inclusion in research programs [32]. They offer 1:1 consultations as well as a growing library of evidence-based toolkits and videos on engagement best practices, including in many of the topics outlined in Table 5 item C; this in turn is enabling NIH-funded researchers to conduct high-quality engaged IS.

Strengths and limitations

Strengths of this research prioritization study include the multiple sources of input beginning organically at the 15th Annual Conference on the Science of Dissemination and Implementation followed by focus groups with scientists with diverse professional and personal backgrounds, and a panel of senior scientists and experienced patient and community advocates. Limitations include that the needs and barriers identified differed by scope and scale (i.e., some are bigger problems than others), making them difficult to rank. Similarly, some of the recommendations on research opportunities were more specific than others. We had feasibility limitations in the number of focus groups that could be conducted; additional focus groups would likely have generated additional needs, barriers, and research opportunities. Finally, while our respondents and panelists came from a range of backgrounds, this study included only professional respondents with a preponderance of engagement scientists. Despite efforts to recruit participants and experts with perspectives from low- and middle-income countries (LMICs), only one focus group participant was from Nigeria and all expert panelists were U.S.-based. Our findings may not reflect the priorities of other parties, including patients, community members, clinicians, and those of researchers and communities in LMICs.

Conclusions

The present study reports on the development of a collection of priority scientific needs and research opportunities related to engagement for implementation scientists and funding agencies to consider. Main conclusions are that research needs exist that relate to engagement processes, outcomes, and structural barriers, even for scientists who are bought in to the value of engaged research. Expert panelists recommended sequential and reinforcing research opportunities that implementation and engagement scientists can tackle together to advance both fields and health equity. Future work should assess insights from broader invested parties, particularly patients and community members.