Mobile health interventions for HIV/STI prevention among youth in low- and middle-income countries (LMICs): a systematic review of studies reporting implementation outcomes

Background Advances and proliferation of technologies such as mobile phones may provide opportunities to improve access to HIV/STI services and reach young people with high risk for HIV and STI. However, the reach, uptake, and sustainability of mobile health (mHealth) HIV/STI interventions targeting young people aged 10–24 years in low- and middle-income countries (LMICs) are largely unknown. To address this gap and to inform implementation science research, a review was conducted to summarize what is known, and what we need to know about implementing mhealth interventions for HIV/STI prevention targeting young people in LMICs. Methods We used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines for this review. Drawing upon Proctor’s eight implementation outcome measures, we evaluated the acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, and sustainability of m-health HIV/STI interventions targeting young people in LMICs. The search was performed from September 2020–January 2021 and updated on March 1, 2021, in Cumulative Index to Nursing and Allied Health Literature (CINAHL), PubMed, SCOPUS, Global Health, and Web of Science. Eligible studies were required to include an HIV/STI prevention outcome, target young people aged 10–24 years, include a comparison/control group, and reporting of atleast one implementation outcome as outlined by Proctor. Results A total of 1386 articles were located, and their titles and abstracts were screened. Of these, 57 full-text articles were reviewed and subsequently, and 11 articles representing 6 unique interventions were included in the systematic review. Acceptability 6 (100%), appropriateness 6 (100%), and feasibility 5(83%) were the most frequently evaluated implementation outcomes. Adoption 2 (33%), fidelity 1 (17%), and cost 1 (17%) were rarely reported; penetration and sustainability were not reported. Conclusions This review contributes to implementation science literature by synthesizing key implementation outcomes of mHealth HIV/STI interventions targeting young people in LMICs. Future research is needed on m-health HIV/STI implementation outcomes, particularly the penetration, cost, and long-term sustainability of these interventions. Doing so will enhance the field’s understanding of the mechanisms by which these interventions lead or do not lead to changes in high HIV/STI risk and vulnerability among young people in LMICs. Supplementary Information The online version contains supplementary material available at 10.1186/s43058-021-00230-w.

Contributions to the literature • With recent calls for efforts to enhance the translation of effective interventions into real-world settings, this systematic review examined the implementation outcomes of mhealth interventions for HIV/STI prevention among young people in LMICs. This study addresses an important gap in knowledge given the paucity of evidence regarding the dissemination and scale-up of mhealth interventions for HIV/STI prevention among young people residing in resource-constrained settings. • This systematic review yielded only six unique comparison-group interventions across eleven articles focused on HIV/STI prevention among young people in LMICs. This suggests that the adoption and delivery of mhealth interventions for HIV/STI prevention among young people in LMICs using rigorous study designs is occurring more gradually than the rapid increase in mobile technologies in the region. • Findings from this systematic review highlight gaps in the reporting of macro-level implementation outcomes such as penetration and sustainability that can impact the long-term effect of interventions on population health. Future studies can help address this measurement gap by providing nuanced measures for these poorly reported long-term implementation outcomes.

Background
In the past two decades, there has been an explosive increase in the ownership of mobile phones globally, with over 7 billion people with a mobile phone subscription in 2018 [1]. The global proliferation of mobile phone use is largely driven by rapid uptake in low-and middleincome countries (LMICs) [2][3][4]. The growing ubiquity and penetration of mobile phone use have led to an increase in the utilization of mobile phones to address public health concerns, broadly referred to as mobile health (mHealth) [5,6]. MHealth interventions have shown some success in expanding access to care and improving existing health interventions [4,[6][7][8]. Among young people, the rapid increase in the use of mobile phone technologies affords additional modalities/opportunities to meet their health needs [9,10]. Human immunodeficiency virus (HIV) and other sexually transmitted infections (STIs) remain significant public health concerns among young people in LMICs [10], suggesting the need for effective strategies to improve access and reach of evidence-based HIV/STI prevention interventions. MHealth interventions may contribute to filling this gap by reaching young people not currently reached by existing HIV/STI interventions.
Previous studies have highlighted ways in which mHealth can promote HIV and STI prevention including serving as reminders for health action, boosters to reinforce sexual risk reduction skills, platforms to promote HIV/STI knowledge, and linkage to HIV/ STI screening services [11][12][13]. Studies evaluating the effectiveness of mhealth interventions have also shown promising results such as increase in HIV testing, condom use, and HIV/STI knowledge among young people [14,15]. A 2017 systematic review on the use of mobile technologies for adolescent sexual and reproductive health in LMICs concluded that interventions delivered through this modality were effective for enhancing HIV and STI prevention [16]. Mhealth interventions were found to improve HIV/STI knowledge and linkage to STI testing services [16]. Overall, existing evidence suggests that mHealth interventions can promote HIV/STI prevention programs/interventions [15] and that the use of mHealth interventions are generally acceptable for young people [15,17]. Despite documented impact and popularity of mHealth interventions for HIV and STI prevention, there is limited evidence demonstrating how-and if-such interventions with reported efficacy have been translated beyond research settings and implemented in real-world settings. Given the growing need to enhance the uptake of innovative approaches, such as mHealth interventions for HIV/STI prevention among youth in LMICs, it is critical to identify implementation components that influence translating these research evidence to real-world settings.
Implementation science research may improve the uptake and reach of m-health HIV/STI interventions targeting young people in LMICs. Implementation science research helps to improve the knowledge of factors that influence the effectiveness of an intervention and strategies needed to accelerate the integration of research findings and research-based innovations into Doing so will enhance the field's understanding of the mechanisms by which these interventions lead or do not lead to changes in high HIV/STI risk and vulnerability among young people in LMICs.
Keywords: Mobile health, Young people, HIV/STI prevention, Implementation outcomes, Mobile phone, Low-and middle-income countries real-world practice settings [18]. Proctor and colleagues [18] implementation outcome framework provides a tool for measuring and operationalizing implementation outcomes-acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, and sustainability. This tool was designed to help evaluate the components and success of intervention implementation efforts, and it is widely used in implementation science research [18]. Evaluating these implementation outcomes is crucial to gain information on the implementation process and to identify potential barriers and facilitators to intervention translation to real-world settings [18,19]. While prior systematic reviews have reported that mHealth interventions can impact HIV and STI prevention [2,20], no published systematic review to our knowledge has examined the implementation outcomes of these interventions. As a result, our study sought to identify and evaluate the implementation outcomes documented in mHealth interventions for HIV and STI prevention among young people in LMICs. These findings may inform the real-world implementation of these interventions particularly among hard-to-reach youth populations in LMICs.

Methods
The systematic review was conducted and reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [21]. The PRISMA checklist is available in Additional file 1. The protocol is registered on PROSPERO, protocol ID: CRD42020196138.

Inclusion and exclusion criteria
Articles were included in the review if: a) Researchers used mobile health (mHealth) and addressed HIV/STI prevention in their intervention b) They were full-text peer-reviewed empirical research c) The study design was a randomized controlled trial (RCT) (with comparison group and random assignment) or quasi-experimental (with comparison groups but no random assignment) or steppedwedge, controlled before-after, or interrupted time series design with a control group d) The intervention was conducted in at least one country classified as LMICs as defined by the World Bank classification of country income groups [22] e) Researchers reported at least one outcome evaluation that assessed the impact of the intervention on at least one or more of the following HIV/STI pre-vention outcomes among young people: HIV/STI  testing, condom use, condom use self-efficacy and  attitudes, condom use intentions, HIV/STI related  knowledge, PrEP use, number of sexual partners, HIV/STI incidence f ) The research targeted adolescents and young people aged 10-24 years. Interventions that were not specific to adolescents and young people but reported separately for this population were eligible for the review. g) Implementation outcome(s) were reported (i.e., acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, and sustainability) [18]

h) Was published in English language
Articles were excluded in the review if: a) No intervention was performed b) The study did not measure any HIV/STI related prevention outcomes c) mHealth tools were used only for data collection or sample recruitment d) The study did not include a control group. We only included studies with comparison groups in the review to minimize the risk of biases associated with non-comparison group study designs e) The study did not report any implementation outcome f ) Were reviews, commentaries, editorials, conference papers, and other non-peer-reviewed publications g) Studies were not available in English No limitation-or restriction-on the year of publication was applied.

Search strategy
The literature search was conducted in September 2020 through January 2021 and repeated March 1, 2021, using the following 5 electronic databases: Cumulative Index to Nursing and Allied Health Literature (CINAHL), PubMed, SCOPUS, Global Health, and Web of Science. The searches were performed using the following list of keywords and mesh terms around the four domains of "HIV and STI prevention, " "mobile technology, " "young people, " and "LMICs. " A detailed search strategy for the PubMed database is provided in Additional file 2. Manual reference searches of prior systematic reviews related to mHealth interventions for STI/HIV prevention were completed for other potentially relevant articles [23][24][25][26][27]. We also examined reference lists of all included articles and relevant reviews [20,24,28] to search for additional studies.

Study selection criteria
Following the search, all identified citations were collated and uploaded into Endnote X8 software, where duplicates were removed. Titles and abstracts were screened by two independent reviewers (UN, CO) against the inclusion and exclusion criteria for the review. Studies that met the inclusion criteria were retrieved in full. The full texts of selected studies were retrieved and assessed in detail against the inclusion criteria by two reviewers (UN, CO). Full texts of studies that did not meet the inclusion criteria were excluded and reasons for the exclusion were documented. Disagreements were resolved through discussion between the two reviewers at each stage of the study selection process. The results of the literature search process are presented in a PRISMA flow diagram.

Data extraction and analysis
Data from the included studies were extracted independently by two reviewers (UN, CO) with a piloted data extraction form. The process was validated by assessing the data extraction form on a small number of randomly selected studies (n=3) that were completed independently by the two reviewers. A third person (TS) evaluated the extracted data using the extraction form from the two reviewers (UN, CO) and discussed discrepancies with them. Data extracted from each study included the year of publication, authors, country of origin, study objective, study design, population and age, participants sample size, intervention description (format, content, setting, mode of delivery), intervention characteristics, outcome measures, and key findings. Implementation outcomes were extracted according to the implementation outcome framework by Proctor and colleagues acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, and sustainability [18] and an adapted data extraction tool used by Ugalde and colleagues [29]. Table 1 provides more details on the definition implementation outcomes for data extraction.
Given the heterogeneity of the studies (variety of intervention components and outcomes measures) included in this review, it was not practical to perform a metaanalysis. Therefore, the extracted data were analyzed using narrative synthesis. Details extracted from the included articles were synthesized using tabulation and textual description [30].
First, we performed a descriptive analysis of the final articles to record key study characteristics such as authors, publication year, study aim, study design, sample size, country of origin, types of intervention, and study outcomes. Second, we reported on implementation outcome measures for the included interventions as defined by Proctor and colleagues [18].

Risk of bias and quality assessment
The Cochrane risk of bias assessment tool was used to assess the risk of selection bias, reporting bias, performance bias, detection bias, and attrition bias in randomized controlled trials (RCTs) [31]. The tool consists of six domains: selection bias, performance bias, detection bias, attrition bias, reporting bias, and other biases [31,32]. Two reviewers (U.N. and C.O) independently assessed the quality of the selected studies. Any discordance in the assignment of quality assessment was resolved by discussion. The risk of bias of the interventions was rated as low, high, or unclear. The Cochrane Collaboration risk of bias assessment tool was used to evaluate the internal validity of the studies included in the review, and no study was excluded from the review due to the risk of bias assessment score. The relevance of articles included in the review was ascertained by the Table 1 Definition of implementation outcomes for data extraction [18,29]

Acceptability
Perception among implementation stakeholders that a given treatment, service, practice, or innovation is agreeable, palatable, or satisfactory

Adoption
The intention, initial decision, or action to try or employ an innovation or evidence-based practice

Appropriateness
The perceived fit, relevance, or compatibility of the innovation or evidence-based practice for a given practice setting, provider, or consumer; and/or perceived fit of the innovation to address a particular issue or problem

Feasibility
The extent to which a new treatment, or an innovation, can be successfully used or carried out within a given agency or setting

Fidelity
The degree to which an intervention was implemented as it was prescribed in the original protocol or as it was intended by the program developers

Cost
The financial impact of an implementation effort

Penetration
The integration of a practice within a service setting and its subsystems

Sustainability
The extent to which a newly implemented treatment is maintained or institutionalized within a service setting ongoing, stable operations  Figure 1 presents a flow chart of the literature search and selection process. We identified a total of 1386 studies through the journal database. Of these, 57 articles were potentially relevant based on their title and abstract. The full text of these 57 articles were assessed for eligibility, of which 46 were excluded. The remaining 11 articles were finally included in the review. Reviewing the reference lists of the included papers and previous systematic reviews in the area did not result in additional articles. In total, 11 articles representing 6 unique interventions were included in the review.

Study characteristics
An overview of the study characteristics of the selected studies is presented in Table 2. Four (67%) of the interventions were conducted in sub-Saharan Africa: 2 (33%) in Kenya [33,34], 1 (17%) in Ghana [15], and 1 (17%) in Uganda [35]. Two (33%) interventions were in East Asia: both of them in China [36,37]. The average number of participants included in the studies was 594, ranging from 60 [33] to 1337 [36]. The mHealth components across the six interventions were delivered using three modalities: (1) as mobile applications, (2) as phone-based short message services (SMS), and (3) as web-based application. Specifically, two interventions used mobile phone applications to provide HIV/STI prevention services and information [33,37]. One of the intervention was delivered as a narrativebased game for android smartphones [33] and one used    WeChat-a messaging mobile application [37]. WeChat was used to provide messages and referrals to health services. Two other interventions used web-based applications to deliver intervention components [35,36]. Lou et al. [36] utilized a web-based application to offer sexual and reproductive health knowledge, service information, counseling, and discussion to study participants and Ybarra and colleagues [35] utilized a web-based application called CyberSenga to provide online module information on HIV prevention and risk-reduction education. Additionally, two of the interventions used SMS technology to deliver intervention components [15,34]. Five of the six (83%) interventions in the review were evaluated through the use of RCT and [15,[33][34][35]37], one (17%) using a quasi-experiment [36]. The comparison conditions for five of the interventions were standard of care, access to usual HIV/STI education at school, community, or home [15,[33][34][35][36]. For the study focused on promoting HIV self-testing (HIVST) in China, the control condition was the absence of the Wechat mobile application which was available to participants in the intervention group [37]. Participants in both the control group and the intervention group received oral HIVST kits for this intervention [37].

Evidence on the effect of mHealth intervention on HIV/STI prevention
Three of the interventions primarily targeted increasing condom use [33,35,37], two targeted improving sexual and reproductive health knowledge [15,36], and one study targeted delaying sexual intercourse after male circumcision [34]. The various interventions in general focused on different measures of HIV/STI prevention. This included increase in HIV/STI knowledge, reduction in risky sexual behaviors (e.g., delay in sexual intercourse after male circumcision, increase condom use, delay sexual debut), and increase HIV self-testing.

Implementation outcomes
We identified studies that reported implementation outcomes as defined by Proctor et al. [18]. Figure 2 displays the frequency of which these implementation outcomes were measured across the six interventions included in this review. The most commonly evaluated implementation outcomes across the interventions were acceptability (100%) and appropriateness (100%), followed by feasibility (83.3%). Across all interventions, adoption (33.3%), fidelity (16.7%), and cost (16.7%) were the least measured implementation outcomes. Penetration and sustainability were not evaluated in any of the interventions included in this review. Reporting of implementation outcomes is at the intervention-level for the results provided.

Acceptability
Acceptability was measured and reported in all six (100%) interventions included in the review [15,[33][34][35][36][37]. Acceptability was identified as an implementation outcome in each study if the study participants indicated satisfaction with the technology or based on their experience with the intervention process. All the interventions were rated to be highly acceptable by study participants, stakeholders, and from reports by the researchers.
Acceptability was evaluated based on data collected on intervention satisfaction from participants' perspectives for the six interventions. Two of the interventions explicitly measured intervention acceptability using Likert  scale survey questionnaires and qualitative feedbacks from study participants [15,33]. Odeny and colleagues measured acceptability based on the success of SMS used to deliver intervention components to study participants [34]. Intervention acceptability in the studies by Odeny and colleagues in Kenya [34,38], and Lou and colleagues in China [36] were determined through pilot studies among the target population prior to the intervention implementation. In the intervention by Rokicki et al. [15], acceptability was assessed based on high levels of participants' engagement in the study. In the intervention in Kenya by Winskell and colleagues [33], study participants found the games to be valuable. Specifically, all participants indicated that they learned a lot from the game and that the information acquired would be very useful in their future. Additionally, 97% of the participants indicated that they would tell their friends to play the game. All the intervention reported either involved participants and/or key stakeholders in the intervention development process. Lou et al. [36] included participants' suggestions in the website development process to ensure that it was attractive to the target population (young students). In the intervention by Ybarra et al. [35], young people were involved in the intervention development. They provided feedback on the mobile application usability prior to deployment. The intervention by Zhu et al. [37] engaged participants through interviews to explore users' preferences concerning content for HIV self-testing promotion. In addition, in the development of Tumaini narrativebased game application adolescents and their parents were convened to provide feedback on the game components to ensure they were suitable for and acceptable to young people in Kenya [33]. To enhance the acceptability of the SMS quiz intervention for adolescents in Ghana, adolescents were engaged in focus group discussions to understand their priority sexual and reproductive health needs. This informed the message content for the intervention, which was further approved by the Health Promotion Unit at the Ghana Health Service [14,15]. Overall participants and stakeholders' feedback and suggestions were incorporated in the intervention components and implementation to enhance intervention acceptability.

Adoption
Adoption defined here as the intention, initial decision, and action to take up or utilize the mHealth intervention was reported in two (33%) interventions [15,35]. This was only measured at the participants' level and not at the setting level. Rokicki & Fink [15] evaluated intervention adoption based on high participants' engagement. Specifically, they measured the total number of times participants replied to the weekly message quiz questions posed in the interactive mobile phone quiz game. Ybarra et al. [35] measured adoption by the number of participants who completed the intervention. The intervention was completed by 95% of participants in the intervention group. For the two interventions that reported on adoption, this was assessed to be high.

Appropriateness
Appropriateness defined here as the perceived fit, relevance, usefulness, and compatibility of mHealth technology with study participants' settings and needs was measured in all six (100%) interventions [15,[33][34][35][36][37]. Three of the interventions employed strategies to target specific youth populations who may have higher risks for HIV/STI infections compared to their peers. For instance, the intervention by Zhu and colleagues [37] utilized purposive sampling to target men who have sex men (MSM) who were at increased risk for HIV. Similarly, the interventions by Lou et al. [36] and Winskell et al. 2018 [33] recruited study participants using purposive sampling to maximize reach to their target audience. Appropriateness was mainly evaluated based on participants' feedback. In the intervention by Winskell et al. [33], study participants reported in surveys that the intervention was useful and applicable to preventing HIV and other STIs. They futher indicated that they acquired useful information from the study which were relevant to their daily lives and future. Similarly, participants found the CyberSenga to be acceptable and not contradictory to local norms in Uganda in most cases, however concerns on the the program including a lot of discussion on sex and condoms were raised [35]. Particularly, about 70% of the participants stated that the CyberSenga program "talked too much about sex and condom use". However, evaluation of the program found that information about condom use and sex education was not confusing and contradictory to youth participants who were abstinent [35]. They information was helpful to build risk-reduction skills. The CyberSenga intervention by Ybarra and colleagues [39,40] also measured intervention appropriateness and fit based on the availability of internet computer access or electricity in the classrooms-the intervention sites. While the intervention components were evaluated to be appropriate in two intervention sites, the fit was limited for sites that did not have Internet or computer access, or electricity. The researchers created mobile cafés for participants in these schools to have access to the intervention components. Overall, their was mixed-reporting of appropriateness across the interventions.

Feasibility
Feasibility was measured in five (83%) of the interventions [15,[33][34][35]37] and defined here as the extent to which the intervention was successfully implemented. This implementation outcome was evaluated based on feedback from study participants and process evaluation of the intervention implementation. For instance, the intervention by Ybarra et al. [35] assessed intervention feasibility using field notes and process measures of the intervention implementation. The intervention was reported to be feasible due to high participation rate. The participation rate for the intervention remained high despite some interruptions in interventions due to conflicting school and participants' schedules. Given that the intervention CyberSenga program required Internet, the absence of Internet interruptions during intervention delivery was reported as an indicator of high intervention feasibility. Zhu et al. [37] measured intervention feasibility based on increased uptake of HIV testing among study participants. In addition, the five interventions all reported on participants eligibility criteria, tscreening process, and the number of participants who finally enrolled in the interventions. These are also important indicators to assess intervention feasibility. Overall, the five interventions were evaluated to be suitable and practical for everyday use for participants.

Fidelity
Only one (17%) intervention explicitly evaluated intervention fidelity [14]. Fidelity was measured by recording the delivery of sexual reproductive health text messages by staff, to ensure consistency in the messages for intervention participants.

Penetration
Penetration which is defined as the spread within an eligible population or level of institutionalization [18] was not evaluated in any of the interventions (0%).

Cost
Cost was reported in one of the interventions [15], defined as the financial impact needed for the delivery of intervention components [18]. The text-messaging intervention for adolescent girls in Ghana reported that the marginal cost for the interactive program per participant was US$1.91 and for the unidirectional program was US$0.30 [15]. The unidirectional intervention sent participants text messages with reproductive health information. The interactive intervention engaged adolescents in text-messaging reproductive health quizzes [15]. The authors explained this marginal cost to be inexpensive and have the potential to reach a large and diverse population [15]. However, additional information was not provided on how the cost was computed or what the cost consisted of.

Sustainability
None of the intervention evaluated sustainability, which is defined as the extent to which an intervention can be maintained, routinized, or institutionalized by a provider or facility [18].

Quality of evidence
The quality assessment of the selected articles is reported in Table 3. Overall, the methodological rigor of the included interventions was high. The level of bias did not vary widely, the majority of the interventions (86%) had a bias rate of 14.3%, and only one of the interventions had a bias rate of 28.6%. The major strengths of the interventions included the use a random selection process and random assignment of participants to the intervention components, detailed description of intervention and participants' characteristics. The low attrition reported across the interventions was another major strength. The major limitation of the interventions was that study participants were not blinded to the intervention allocation. According to the intervention by Rokicki and colleagues [15], it was not feasible and practical to blind the study participants to the intervention given the overt participation nature of the mhealth intervention. In the intervention by Winskell and colleagues [33], the researcher was blinded to the allocation while study participants were revealed to their assignment as randomization.

Discussion
The current study examined the extent to which mHealth interventions for HIV/STI prevention among young people in LMICs reported on implementation outcomes: acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, and sustainability. Findings from this review show variations in the evaluation of these implementation outcomes across the six interventions included in the review. At least one implementation outcome was reported in all six interventions, with each intervention reporting between two to five implementation outcomes. These results suggest that although LMICs are experiencing exponential technological growth with young people increasingly having access to mobile phones, implementation science research in the area of mHealth interventions for HIV/STI prevention in the region may be at a nascent phase and require additional work. We found several similarities when comparing our results to previous systematic reviews on implementation outcomes. Similar to previous reviews on implementation outcomes in LMICs, acceptability, appropriateness, and feasibility were the most frequently reported implementation outcomes [41][42][43]. All six interventions in the review reported high intervention acceptability. This was largely evaluated based on self-reports and discussions with study participants and target populations. In addition, stakeholders beyond the study participants such as parents and community stakeholders were also involved to evaluate intervention acceptability. For instance, the interventions by Winskell et al. [33] and Rokicki et al. [15] involved other key stakeholders who provided valuable feedback to optimize for the acceptability and suitability of the intervention components. For the Tumaini smartphone application [33], adolescents and their parents were engaged to measure intervention acceptability and in the Rokicki study [15] government stakeholders were also involved to review and approve the intervention message contents.
Further, there were also mixed evidence on the appropriateness of the interventions. While most of the studies reported that mhealth HIV/STI prevention interventions were relevant to and suitable for young people, participants in the Tumaini intervention in Kenya found some components of the intervention to be uncomfortable and not age appropriate [44]. Similar concerns on the discussions on sex and condom use were raised for the Cyber-Senga program in Uganda [35]. This suggests the need to ensure that interventions are tailored to be contextually and developmentally appropriate among target populations while developing and implementing intervention contents geared towards HIV/STI prevention among young people. The literature on the adaptation of HIV/ STI prevention interventions have highlighted the importance of considering local contexts and cultures to ensure the interventions are culturally and developmentally appropriate [45][46][47]. Since culture and social norms play a significant role in the acceptance of intervention content around sensitive topics related to sexual practices and sexual risk-reduction practices such as condom use [48][49][50], these considerations are critical to enhancing intervention adoption and acceptance [51,52].
In addition, adoption and fidelity were minimally reported in the included interventions. Adoption was measured using various metrics across the interventions. The intervention by Rokicki & Fink [15] measured adoption based on participants' engagement in the intervention through weekly responses to the mobile-based SMS preventive HIV and STI quizzes while Ybarra et al. [35] evaluated adoption as intervention components completion rate. This finding is consistent with previous studies, suggesting variations in measuring, reporting, and analyzing adoption [18,53]. However, none of the studies examined setting-level adoption metrics such as location or practice setting readiness or intent-to-use for mHealth interventions to promote HIV/STI prevention. While the information on participant-level adoption are useful, it does not provide nuanced and robust understanding of setting-level intent to try or use m-Health interventions. Future studies should examine setting-level adoption measures, to delineate factors associated with the uptake of the interventions by health facilities, staff, or communities in LMICs. Setting-level information in addition to end-user input provides robust details necessary for designing and adapting interventions to be context-specific [54]. In addition, there was low reporting on intervention fidelity or descriptions on how the interventions are implemented which are crucial information for ensuring the successful replication or adoption of these interventions to maximize effectiveness and public health gains [55]. These gaps are in congruence with reviews of interventions in LMICs, which calls for more robust and consistent measurement of adoption [41,42,56] and fidelity [42]. Future reporting of mHealth interventions for young people in LMICs should consider providing detailed fidelity measures or information to provide structure for effective planning, implementation, and evaluation of interventions. As implementation research advances in the area of mhealth applications in LMICs, it is critical to develop a methodological framework that provides a rigorous measurement of fidelity dimensions such as consistency, satisfaction, and quality can be explored [57]. Such implementation science methodologies may provide more nuanced and reliable measures of fidelity that can be adapted for each setting. Findings from this review also highlight a persistent gap in the measurement of penetration and sustainability, as none of the studies included in the review reported on these implementation outcomes. These implementation outcomes are important broader contextual determinants of policies and strategies for the integration of mHealth interventions in health settings [58][59][60]. The lack of reporting on penetration and sustainability may be as a result of most studies focusing on preliminary efficacy/effectiveness of these m-health interventions. Nonetheless, to optimize the translation of interventions to real-world settings, it is imperative to assess these implementation outcomes. Penetration and sustainability measure and illuminate practice-level and contextual factors that enhance opportunities for integration of interventions into practice and scale-up [61,62]. Also, only one of the interventions reported on intervention cost, evidence on the cost, and cost-effectiveness of mobile health interventions are essential to justify scale-up and allocations of funds in regions where resources are scarce [63]. Reporting on cost should be rigorous to account for the financial impact of the mhealth implementation from the cost of mhealth platform to the cost of staff. This should also highlight startup and sustainability costs.
This review must also be seen in the light of some limitations. First, it is possible our search strategy may have excluded potentially eligible articles. To minimize this risk, articles were retrieved from multiple electronic databases and supplemented by manual search and reference list checking of included articles. Second, we limited our review to only RCTs and quasi-experimental studies. Observation studies and non-control studies may provide important information useful for understanding the implementation outcomes of mHealth interventions for HIV/STI prevention among young people in LMICs [64]. Our inclusion criteria may have also resulted in omitting pre-implementation acceptability or feasibility assessments that were not accompanied by RCTs or quasi-experimental studies. However, we opted for interventions that had some comparison groups to minimize the risk of bias associated with non-comparison group study designs. Third, this review assessed the implementation potential of intervention based on the reporting on the published intervention, but this may not mean that it has not been translated into practice. Studies may show limited implementation potential according to reported information in the articles but may have been successfully implemented into practice [29]. Nonetheless, the data extraction and validation by the two reviewers showed no discrepancy in the allocation of implementation outcomes for each study.
Despite the aforementioned limitations, there are several strengths to this systematic review. To our knowledge, this is the first review to elucidate implementation outcomes of mHealth interventions for HIV/STI preventions among young people in LMICs. Findings from this review can inform mHealth implementations in the region, especially for sustainable and large-scale implementations. In addition, key stakeholders and researchers seeking to understand implementation outcomes of mHealth intervention can benefit from the findings of this review [65], as future research questions on how to improve measurements of implementation outcomes and decision-making on effective means to translate evidence from mHealth interventions to practice can be identified.
This systematic review also has implications for implementation science and practice. Findings underscore the importance of documenting implementation outcomes in more detail to inform other researchers interested in implementing HIV/STI prevention mHealth interventions for young people in LIMICs. This review highlights that gaps exist in this area of research. Assessment of implementation outcomes provides a solid framework to assess the implementation of interventions and offer a unique contribution to the field of implementation science. Findings from this study also provide insights into strategies for integrating mHealth interventions for HIV/ STI prevention for young people in real-world settings. Through highlighting measured implementation outcomes, we aim to provide evidence to assist researchers, practitioners, and policymakers in the process of