Skip to main content
Download PDF
Download ePub
  • Systematic review
  • Open access
  • Published:

Mobile health interventions for HIV/STI prevention among youth in low- and middle-income countries (LMICs): a systematic review of studies reporting implementation outcomes

Implementation Science Communications volume 2, Article number: 126 (2021) Cite this article

Abstract

Background

Advances and proliferation of technologies such as mobile phones may provide opportunities to improve access to HIV/STI services and reach young people with high risk for HIV and STI. However, the reach, uptake, and sustainability of mobile health (mHealth) HIV/STI interventions targeting young people aged 10–24 years in low- and middle-income countries (LMICs) are largely unknown. To address this gap and to inform implementation science research, a review was conducted to summarize what is known, and what we need to know about implementing mhealth interventions for HIV/STI prevention targeting young people in LMICs.

Methods

We used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines for this review. Drawing upon Proctor’s eight implementation outcome measures, we evaluated the acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, and sustainability of  m-health HIV/STI interventions targeting young people in LMICs. The search was performed from September 2020–January 2021 and updated on March 1, 2021, in Cumulative Index to Nursing and Allied Health Literature (CINAHL), PubMed, SCOPUS, Global Health, and Web of Science. Eligible studies were required to include an HIV/STI prevention outcome, target young people aged 10–24 years, include a comparison/control group, and reporting of atleast one implementation outcome as outlined by Proctor.

Results

A total of 1386 articles were located, and their titles and abstracts were screened. Of these, 57 full-text articles were reviewed and subsequently, and 11 articles representing 6 unique interventions were included in the systematic review. Acceptability 6 (100%), appropriateness 6 (100%), and feasibility 5(83%) were the most frequently evaluated implementation outcomes. Adoption 2 (33%), fidelity 1 (17%), and cost 1 (17%) were rarely reported; penetration and sustainability were not reported.

Conclusions

This review contributes to implementation science literature by synthesizing key implementation outcomes of mHealth HIV/STI interventions targeting young people in LMICs. Future research is needed on m-health HIV/STI implementation outcomes, particularly the penetration, cost, and long-term sustainability of these interventions. Doing so will enhance the field’s understanding of the mechanisms by which these interventions lead or do not lead to changes in high HIV/STI risk and vulnerability among young people in LMICs.

Peer Review reports

Background

In the past two decades, there has been an explosive increase in the ownership of mobile phones globally, with over 7 billion people with a mobile phone subscription in 2018 [1]. The global proliferation of mobile phone use is largely driven by rapid uptake in low- and middle-income countries (LMICs) [2,3,4]. The growing ubiquity and penetration of mobile phone use have led to an increase in the utilization of mobile phones to address public health concerns, broadly referred to as mobile health (mHealth) [5, 6].

MHealth interventions have shown some success in expanding access to care and improving existing health interventions [4, 6,7,8]. Among young people, the rapid increase in the use of mobile phone technologies affords additional modalities/opportunities to meet their health needs [9, 10]. Human immunodeficiency virus (HIV) and other sexually transmitted infections (STIs) remain significant public health concerns among young people in LMICs [10], suggesting the need for effective strategies to improve access and reach of evidence-based HIV/STI prevention interventions. MHealth interventions may contribute to filling this gap by reaching young people not currently reached by existing HIV/STI interventions.

Previous studies have highlighted ways in which mHealth can promote HIV and STI prevention including serving as reminders for health action, boosters to reinforce sexual risk reduction skills, platforms to promote HIV/STI knowledge, and linkage to HIV/STI screening services [11,12,13]. Studies evaluating the effectiveness of mhealth interventions have also shown promising results such as increase in HIV testing, condom use, and HIV/STI knowledge among young people [14, 15]. A 2017 systematic review on the use of mobile technologies for adolescent sexual and reproductive health in LMICs concluded that interventions delivered through this modality were effective for enhancing HIV and STI prevention [16]. Mhealth interventions were found to improve HIV/STI knowledge and linkage to STI testing services [16]. Overall, existing evidence suggests that mHealth interventions can promote HIV/STI prevention programs/interventions [15] and that the use of mHealth interventions are generally acceptable for young people [15, 17]. Despite documented impact and popularity of mHealth interventions for HIV and STI prevention, there is limited evidence demonstrating how—and if—such interventions with reported efficacy have been translated beyond research settings and implemented in real-world settings. Given the growing need to enhance the uptake of innovative approaches, such as mHealth interventions for HIV/STI prevention among youth in LMICs, it is critical to identify implementation components that influence translating these research evidence to real-world settings.

Implementation science research may improve the uptake and reach of m-health HIV/STI interventions targeting young people in LMICs. Implementation science research helps to improve the knowledge of factors that influence the effectiveness of an intervention and strategies needed to accelerate the integration of research findings and research-based innovations into real-world practice settings [18]. Proctor and colleagues [18] implementation outcome framework provides a tool for measuring and operationalizing implementation outcomes—acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, and sustainability. This tool was designed to help evaluate the components and success of intervention implementation efforts, and it is widely used in implementation science research [18]. Evaluating these implementation outcomes is crucial to gain information on the implementation process and to identify potential barriers and facilitators to intervention translation to real-world settings [18, 19].

While prior systematic reviews have reported that mHealth interventions can impact HIV and STI prevention [2, 20], no published systematic review to our knowledge has examined the implementation outcomes of these interventions. As a result, our study sought to identify and evaluate the implementation outcomes documented in mHealth interventions for HIV and STI prevention among young people in LMICs. These findings may inform the real-world implementation of these interventions particularly among hard-to-reach youth populations in LMICs.

Methods

The systematic review was conducted and reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [21]. The PRISMA checklist is available in Additional file 1. The protocol is registered on PROSPERO, protocol ID: CRD42020196138.

Inclusion and exclusion criteria

Articles were included in the review if:

  1. a)

    Researchers used mobile health (mHealth) and addressed HIV/STI prevention in their intervention

  2. b)

    They were full-text peer-reviewed empirical research

  3. c)

    The study design was a randomized controlled trial (RCT) (with comparison group and random assignment) or quasi-experimental (with comparison groups but no random assignment) or stepped-wedge, controlled before-after, or interrupted time series design with a control group

  4. d)

    The intervention was conducted in at least one country classified as LMICs as defined by the World Bank classification of country income groups [22]

  5. e)

    Researchers reported at least one outcome evaluation that assessed the impact of the intervention on at least one or more of the following HIV/STI prevention outcomes among young people: HIV/STI testing, condom use, condom use self-efficacy and attitudes, condom use intentions, HIV/STI related knowledge, PrEP use, number of sexual partners, HIV/STI incidence

  6. f)

    The research targeted adolescents and young people aged 10-24 years. Interventions that were not specific to adolescents and young people but reported separately for this population were eligible for the review.

  7. g)

    Implementation outcome(s) were reported (i.e., acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, and sustainability) [18]

  8. h)

    Was published in English language

Articles were excluded in the review if:

  1. a)

    No intervention was performed

  2. b)

    The study did not measure any HIV/STI related prevention outcomes

  3. c)

    mHealth tools were used only for data collection or sample recruitment

  4. d)

    The study did not include a control group. We only included studies with comparison groups in the review to minimize the risk of biases associated with non-comparison group study designs

  5. e)

    The study did not report any implementation outcome

  6. f)

    Were reviews, commentaries, editorials, conference papers, and other non-peer-reviewed publications

  7. g)

    Studies were not available in English

No limitation- or restriction- on the year of publication was applied.

Search strategy

The literature search was conducted in September 2020 through January 2021 and repeated March 1, 2021, using the following 5 electronic databases: Cumulative Index to Nursing and Allied Health Literature (CINAHL), PubMed, SCOPUS, Global Health, and Web of Science. The searches were performed using the following list of keywords and mesh terms around the four domains of “HIV and STI prevention,” “mobile technology,” “young people,” and “LMICs.” A detailed search strategy for the PubMed database is provided in Additional file 2. Manual reference searches of prior systematic reviews related to mHealth interventions for STI/HIV prevention were completed for other potentially relevant articles [23,24,25,26,27]. We also examined reference lists of all included articles and relevant reviews [20, 24, 28] to search for additional studies.

Study selection criteria

Following the search, all identified citations were collated and uploaded into Endnote X8 software, where duplicates were removed. Titles and abstracts were screened by two independent reviewers (UN, CO) against the inclusion and exclusion criteria for the review. Studies that met the inclusion criteria were retrieved in full. The full texts of selected studies were retrieved and assessed in detail against the inclusion criteria by two reviewers (UN, CO). Full texts of studies that did not meet the inclusion criteria were excluded and reasons for the exclusion were documented. Disagreements were resolved through discussion between the two reviewers at each stage of the study selection process. The results of the literature search process are presented in a PRISMA flow diagram.

Data extraction and analysis

Data from the included studies were extracted independently by two reviewers (UN, CO) with a piloted data extraction form. The process was validated by assessing the data extraction form on a small number of randomly selected studies (n=3) that were completed independently by the two reviewers. A third person (TS) evaluated the extracted data using the extraction form from the two reviewers (UN, CO) and discussed discrepancies with them.

Data extracted from each study included the year of publication, authors, country of origin, study objective, study design, population and age, participants sample size, intervention description (format, content, setting, mode of delivery), intervention characteristics, outcome measures, and key findings. Implementation outcomes were extracted according to the implementation outcome framework by Proctor and colleagues acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, and sustainability [18] and an adapted data extraction tool used by Ugalde and colleagues [29]. Table 1 provides more details on the definition implementation outcomes for data extraction.

Table 1 Definition of implementation outcomes for data extraction [18, 29]
Full size table

Given the heterogeneity of the studies (variety of intervention components and outcomes measures) included in this review, it was not practical to perform a meta-analysis. Therefore, the extracted data were analyzed using narrative synthesis. Details extracted from the included articles were synthesized using tabulation and textual description [30].

First, we performed a descriptive analysis of the final articles to record key study characteristics such as authors, publication year, study aim, study design, sample size, country of origin, types of intervention, and study outcomes. Second, we reported on implementation outcome measures for the included interventions as defined by Proctor and colleagues [18].

Risk of bias and quality assessment

The Cochrane risk of bias assessment tool was used to assess the risk of selection bias, reporting bias, performance bias, detection bias, and attrition bias in randomized controlled trials (RCTs) [31]. The tool consists of six domains: selection bias, performance bias, detection bias, attrition bias, reporting bias, and other biases [31, 32]. Two reviewers (U.N. and C.O) independently assessed the quality of the selected studies. Any discordance in the assignment of quality assessment was resolved by discussion. The risk of bias of the interventions was rated as low, high, or unclear. The Cochrane Collaboration risk of bias assessment tool was used to evaluate the internal validity of the studies included in the review, and no study was excluded from the review due to the risk of bias assessment score. The relevance of articles included in the review was ascertained by the study selection process according to the pre-defined inclusion and exclusion criteria.

Results

Literature search results

Figure 1 presents a flow chart of the literature search and selection process. We identified a total of 1386 studies through the journal database. Of these, 57 articles were potentially relevant based on their title and abstract. The full text of these 57 articles were assessed for eligibility, of which 46 were excluded. The remaining 11 articles were finally included in the review. Reviewing the reference lists of the included papers and previous systematic reviews in the area did not result in additional articles. In total, 11 articles representing 6 unique interventions were included in the review.

Fig. 1
figure 1

Flow diagram of the search strategy. 11 articles representing 6 interventions were included in the review

Full size image

Study characteristics

An overview of the study characteristics of the selected studies is presented in Table 2. Four (67%) of the interventions were conducted in sub-Saharan Africa: 2 (33%) in Kenya [33, 34], 1 (17%) in Ghana [15], and 1 (17%) in Uganda [35]. Two (33%) interventions were in East Asia: both of them in China [36, 37]. The average number of participants included in the studies was 594, ranging from 60 [33] to 1337 [36].

Table 2 Characteristics of included 6 interventions across 11 articles included in the review
Full size table

The mHealth components across the six interventions were delivered using three modalities: (1) as mobile applications, (2) as phone-based short message services (SMS), and (3) as web-based application. Specifically, two interventions used mobile phone applications to provide HIV/STI prevention services and information [33, 37]. One of the intervention was delivered as a narrative-based game for android smartphones [33] and one used WeChat—a messaging mobile application [37]. WeChat was used to provide messages and referrals to health services. Two other interventions used web-based applications to deliver intervention components [35, 36]. Lou et al. [36] utilized a web-based application to offer sexual and reproductive health knowledge, service information, counseling, and discussion to study participants and Ybarra and colleagues [35] utilized a web-based application called CyberSenga to provide online module information on HIV prevention and risk-reduction education. Additionally, two of the interventions used SMS technology to deliver intervention components [15, 34].

Five of the six (83%) interventions in the review were evaluated through the use of RCT and [15, 33,34,35, 37], one (17%) using a quasi-experiment [36]. The comparison conditions for five of the interventions were standard of care, access to usual HIV/STI education at school, community, or home [15, 33,34,35,36]. For the study focused on promoting HIV self-testing (HIVST) in China, the control condition was the absence of the Wechat mobile application which was available to participants in the intervention group [37]. Participants in both the control group and the intervention group received oral HIVST kits for this intervention [37].

Evidence on the effect of mHealth intervention on HIV/STI prevention

Three of the interventions primarily targeted increasing condom use [33, 35, 37], two targeted improving sexual and reproductive health knowledge [15, 36], and one study targeted delaying sexual intercourse after male circumcision [34]. The various interventions in general focused on different measures of HIV/STI prevention. This included increase in HIV/STI knowledge, reduction in risky sexual behaviors (e.g., delay in sexual intercourse after male circumcision, increase condom use, delay sexual debut), and increase HIV self-testing.

Implementation outcomes

We identified studies that reported implementation outcomes as defined by Proctor et al. [18]. Figure 2 displays the frequency of which these implementation outcomes were measured across the six interventions included in this review. The most commonly evaluated implementation outcomes across the interventions were acceptability (100%) and appropriateness (100%), followed by feasibility (83.3%). Across all interventions, adoption (33.3%), fidelity (16.7%), and cost (16.7%) were the least measured implementation outcomes. Penetration and sustainability were not evaluated in any of the interventions included in this review. Reporting of implementation outcomes is at the intervention-level for the results provided.

Fig. 2
figure 2

Implementation outcomes reported in 6 interventions across 11 articles included in the review. Note: Green Shade denotes that the implementation outcome was measured in the study and orange shade denotes that implementation outcome was not measured in the study

Full size image

Acceptability

Acceptability was measured and reported in all six (100%) interventions included in the review [15, 33,34,35,36,37]. Acceptability was identified as an implementation outcome in each study if the study participants indicated satisfaction with the technology or based on their experience with the intervention process. All the interventions were rated to be highly acceptable by study participants, stakeholders, and from reports by the researchers.

Acceptability was evaluated based on data collected on intervention satisfaction from participants’ perspectives for the six interventions. Two of the interventions explicitly measured intervention acceptability using Likert scale survey questionnaires and qualitative feedbacks from study participants [15, 33]. Odeny and colleagues measured acceptability based on the success of SMS used to deliver intervention components to study participants [34]. Intervention acceptability in the studies by Odeny and colleagues in Kenya [34, 38], and Lou and colleagues in China [36] were determined through pilot studies among the target population prior to the intervention implementation. In the intervention by Rokicki et al. [15], acceptability was assessed based on high levels of participants’ engagement in the study. In the intervention in Kenya by Winskell and colleagues [33], study participants found the games to be valuable. Specifically, all participants indicated that they learned a lot from the game and that the information acquired would be very useful in their future. Additionally, 97% of the participants indicated that they would tell their friends to play the game.

All the intervention reported either involved participants and/or key stakeholders in the intervention development process. Lou et al. [36] included participants’ suggestions in the website development process to ensure that it was attractive to the target population (young students). In the intervention by Ybarra et al. [35], young people were involved in the intervention development. They provided feedback on the mobile application usability prior to deployment. The intervention by Zhu et al. [37] engaged participants through interviews to explore users’ preferences concerning content for HIV self-testing promotion. In addition, in the development of Tumaini narrative-based game application adolescents and their parents were convened to provide feedback on the game components to ensure they were suitable for and acceptable to young people in Kenya [33]. To enhance the acceptability of the SMS quiz intervention for adolescents in Ghana, adolescents were engaged in focus group discussions to understand their priority sexual and reproductive health needs. This informed the message content for the intervention, which was further approved by the Health Promotion Unit at the Ghana Health Service [14, 15]. Overall participants and stakeholders’ feedback and suggestions were incorporated in the intervention components and implementation to enhance intervention acceptability.

Adoption

Adoption defined here as the intention, initial decision, and action to take up or utilize the mHealth intervention was reported in two (33%) interventions [15, 35]. This was only measured at the participants’ level and not at the setting level. Rokicki & Fink [15] evaluated intervention adoption based on high participants’ engagement. Specifically, they measured the total number of times participants replied to the weekly message quiz questions posed in the interactive mobile phone quiz game. Ybarra et al. [35] measured adoption by the number of participants who completed the intervention. The intervention was completed by 95% of participants in the intervention group. For the two interventions that reported on adoption, this was assessed to be high.

Appropriateness

Appropriateness defined here as the perceived fit, relevance, usefulness, and compatibility of mHealth technology with study participants’ settings and needs was measured in all six (100%) interventions [15, 33,34,35,36,37].Three of the interventions employed strategies to target specific youth populations who may have higher risks for HIV/STI infections compared to their peers. For instance, the intervention by Zhu and colleagues [37] utilized purposive sampling to target men who have sex men (MSM) who were at increased risk for HIV. Similarly, the interventions by Lou et al. [36] and Winskell et al. 2018 [33] recruited study participants using purposive sampling to maximize reach to their target audience. Appropriateness was mainly evaluated based on participants’ feedback. In the intervention by Winskell et al. [33], study participants reported in surveys that the intervention was useful and applicable to preventing HIV and other STIs. They futher indicated that they acquired useful information from the study which were relevant to their daily lives and future. Similarly, participants found the CyberSenga to be acceptable and not contradictory to local norms in Uganda in most cases, however concerns on the the program including a lot of discussion on sex and condoms were raised [35]. Particularly, about 70% of the participants stated that the CyberSenga program "talked too much about sex and condom use". However, evaluation of the program found that information about condom use and sex education was not confusing and contradictory to youth participants who were abstinent [35]. They information was helpful to build risk-reduction skills. The CyberSenga intervention by Ybarra and colleagues [39, 40] also measured intervention appropriateness and fit based on the availability of internet computer access or electricity in the classrooms–the intervention sites. While the intervention components were evaluated to be appropriate in two intervention sites, the fit was limited for sites that did not have Internet or computer access, or electricity. The researchers created mobile cafés for participants in these schools to have access to the intervention components. Overall, their was mixed-reporting of appropriateness across the interventions.

Feasibility

Feasibility was measured in five (83%) of the interventions [15, 33,34,35, 37] and defined here as the extent to which the intervention was successfully implemented. This implementation outcome was evaluated based on feedback from study participants and process evaluation of the intervention implementation. For instance, the intervention by Ybarra et al. [35] assessed intervention feasibility using field notes and process measures of the intervention implementation. The intervention was reported to be feasible due to high participation rate. The participation rate for the intervention remained high despite some interruptions in interventions due to conflicting school and participants’ schedules. Given that the intervention CyberSenga program required Internet, the absence of Internet interruptions during intervention delivery was reported as an indicator of high intervention feasibility. Zhu et al. [37] measured intervention feasibility based on increased uptake of HIV testing among study participants. In addition, the five interventions all reported on participants eligibility criteria, tscreening process, and the number of participants who finally enrolled in the interventions. These are also important indicators to assess intervention feasibility. Overall, the five interventions were evaluated to be suitable and practical for everyday use for participants.

Fidelity

Only one (17%) intervention explicitly evaluated intervention fidelity [14]. Fidelity was measured by recording the delivery of sexual reproductive health text messages by staff, to ensure consistency in the messages for intervention participants.

Penetration

Penetration which is defined as the spread within an eligible population or level of institutionalization [18] was not evaluated in any of the interventions (0%).

Cost

Cost was reported in one of the interventions [15], defined as the financial impact needed for the delivery of intervention components [18]. The text-messaging intervention for adolescent girls in Ghana reported that the marginal cost for the interactive program per participant was US$1.91 and for the unidirectional program was US$0.30 [15]. The unidirectional intervention sent participants text messages with reproductive health information. The interactive intervention engaged adolescents in text-messaging reproductive health quizzes [15]. The authors explained this marginal cost to be inexpensive and have the potential to reach a large and diverse population [15]. However, additional information was not provided on how the cost was computed or what the cost consisted of.

Sustainability

None of the intervention evaluated sustainability, which is defined as the extent to which an intervention can be maintained, routinized, or institutionalized by a provider or facility [18].

Quality of evidence

The quality assessment of the selected articles is reported in Table 3. Overall, the methodological rigor of the included interventions was high. The level of bias did not vary widely, the majority of the interventions (86%) had a bias rate of 14.3%, and only one of the interventions had a bias rate of 28.6%. The major strengths of the interventions included the use a random selection process and random assignment of participants to the intervention components, detailed description of intervention and participants’ characteristics. The low attrition reported across the interventions was another major strength. The major limitation of the interventions was that study participants were not blinded to the intervention allocation. According to the intervention by Rokicki and colleagues [15], it was not feasible and practical to blind the study participants to the intervention given the overt participation nature of the mhealth intervention. In the intervention by Winskell and colleagues [33], the researcher was blinded to the allocation while study participants were revealed to their assignment as randomization.

Table 3 Reporting on quality of included interventions (6 interventions reported in 11 articles included in the review
Full size table

Discussion

The current study examined the extent to which mHealth interventions for HIV/STI prevention among young people in LMICs reported on implementation outcomes: acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, and sustainability. Findings from this review show variations in the evaluation of these implementation outcomes across the six interventions included in the review. At least one implementation outcome was reported in all six interventions, with each intervention reporting between two to five implementation outcomes. These results suggest that although LMICs are experiencing exponential technological growth with young people increasingly having access to mobile phones, implementation science research in the area of mHealth interventions for HIV/STI prevention in the region may be at a nascent phase and require additional work.

We found several similarities when comparing our results to previous systematic reviews on implementation outcomes. Similar to previous reviews on implementation outcomes in LMICs, acceptability, appropriateness, and feasibility were the most frequently reported implementation outcomes [41,42,43]. All six interventions in the review reported high intervention acceptability. This was largely evaluated based on self-reports and discussions with study participants and target populations. In addition, stakeholders beyond the study participants such as parents and community stakeholders were also involved to evaluate intervention acceptability. For instance, the interventions by Winskell et al. [33] and Rokicki et al. [15] involved other key stakeholders who provided valuable feedback to optimize for the acceptability and suitability of the intervention components. For the Tumaini smartphone application [33], adolescents and their parents were engaged to measure intervention acceptability and in the Rokicki study [15] government stakeholders were also involved to review and approve the intervention message contents.

Further, there were also mixed evidence on the appropriateness of the interventions. While most of the studies reported that mhealth HIV/STI prevention interventions were relevant to and suitable for young people, participants in the Tumaini intervention in Kenya found some components of the intervention to be uncomfortable and not age appropriate [44]. Similar concerns on the discussions on sex and condom use were raised for the CyberSenga program in Uganda [35]. This suggests the need to ensure that interventions are tailored to be contextually and developmentally appropriate among target populations while developing and implementing intervention contents geared towards HIV/STI prevention among young people. The literature on the adaptation of HIV/STI prevention interventions have highlighted the importance of considering local contexts and cultures to ensure the interventions are culturally and developmentally appropriate [45,46,47]. Since culture and social norms play a significant role in the acceptance of intervention content around sensitive topics related to sexual practices and sexual risk-reduction practices such as condom use [48,49,50], these considerations are critical to enhancing intervention adoption and acceptance [51, 52].

In addition, adoption and fidelity were minimally reported in the included interventions. Adoption was measured using various metrics across the interventions. The intervention by Rokicki & Fink [15] measured adoption based on participants’ engagement in the intervention through weekly responses to the mobile-based SMS preventive HIV and STI quizzes while Ybarra et al. [35] evaluated adoption as intervention components completion rate. This finding is consistent with previous studies, suggesting variations in measuring, reporting, and analyzing adoption [18, 53]. However, none of the studies examined setting-level adoption metrics such as location or practice setting readiness or intent-to-use for mHealth interventions to promote HIV/STI prevention. While the information on participant-level adoption are useful, it does not provide nuanced and robust understanding of setting-level intent to try or use m-Health interventions. Future studies should examine setting-level adoption measures, to delineate factors associated with the uptake of the interventions by health facilities, staff, or communities in LMICs. Setting-level information in addition to end-user input provides robust details necessary for designing and adapting interventions to be context-specific [54]. In addition, there was low reporting on intervention fidelity or descriptions on how the interventions are implemented which are crucial information for ensuring the successful replication or adoption of these interventions to maximize effectiveness and public health gains [55]. These gaps are in congruence with reviews of interventions in LMICs, which calls for more robust and consistent measurement of adoption [41, 42, 56] and fidelity [42]. Future reporting of mHealth interventions for young people in LMICs should consider providing detailed fidelity measures or information to provide structure for effective planning, implementation, and evaluation of interventions. As implementation research advances in the area of mhealth applications in LMICs, it is critical to develop a methodological framework that provides a rigorous measurement of fidelity dimensions such as consistency, satisfaction, and quality can be explored [57]. Such implementation science methodologies may provide more nuanced and reliable measures of fidelity that can be adapted for each setting.

Findings from this review also highlight a persistent gap in the measurement of penetration and sustainability, as none of the studies included in the review reported on these implementation outcomes. These implementation outcomes are important broader contextual determinants of policies and strategies for the integration of mHealth interventions in health settings [58,59,60]. The lack of reporting on penetration and sustainability may be as a result of most studies focusing on preliminary efficacy/effectiveness of these m-health interventions. Nonetheless, to optimize the translation of interventions to real-world settings, it is imperative to assess these implementation outcomes. Penetration and sustainability measure and illuminate practice-level and contextual factors that enhance opportunities for integration of interventions into practice and scale-up [61, 62]. Also, only one of the interventions reported on intervention cost, evidence on the cost, and cost-effectiveness of mobile health interventions are essential to justify scale-up and allocations of funds in regions where resources are scarce [63]. Reporting on cost should be rigorous to account for the financial impact of the mhealth implementation from the cost of mhealth platform to the cost of staff. This should also highlight startup and sustainability costs.

This review must also be seen in the light of some limitations. First, it is possible our search strategy may have excluded potentially eligible articles. To minimize this risk, articles were retrieved from multiple electronic databases and supplemented by manual search and reference list checking of included articles. Second, we limited our review to only RCTs and quasi-experimental studies. Observation studies and non-control studies may provide important information useful for understanding the implementation outcomes of mHealth interventions for HIV/STI prevention among young people in LMICs [64]. Our inclusion criteria may have also resulted in omitting pre-implementation acceptability or feasibility assessments that were not accompanied by RCTs or quasi-experimental studies. However, we opted for interventions that had some comparison groups to minimize the risk of bias associated with non-comparison group study designs. Third, this review assessed the implementation potential of intervention based on the reporting on the published intervention, but this may not mean that it has not been translated into practice. Studies may show limited implementation potential according to reported information in the articles but may have been successfully implemented into practice [29]. Nonetheless, the data extraction and validation by the two reviewers showed no discrepancy in the allocation of implementation outcomes for each study.

Despite the aforementioned limitations, there are several strengths to this systematic review. To our knowledge, this is the first review to elucidate implementation outcomes of mHealth interventions for HIV/STI preventions among young people in LMICs. Findings from this review can inform mHealth implementations in the region, especially for sustainable and large-scale implementations. In addition, key stakeholders and researchers seeking to understand implementation outcomes of mHealth intervention can benefit from the findings of this review [65], as future research questions on how to improve measurements of implementation outcomes and decision-making on effective means to translate evidence from mHealth interventions to practice can be identified.

This systematic review also has implications for implementation science and practice. Findings underscore the importance of documenting implementation outcomes in more detail to inform other researchers interested in implementing HIV/STI prevention mHealth interventions for young people in LIMICs. This review highlights that gaps exist in this area of research. Assessment of implementation outcomes provides a solid framework to assess the implementation of interventions and offer a unique contribution to the field of implementation science. Findings from this study also provide insights into strategies for integrating mHealth interventions for HIV/STI prevention for young people in real-world settings. Through highlighting measured implementation outcomes, we aim to provide evidence to assist researchers, practitioners, and policymakers in the process of planning, reporting, and selecting m-health HIV/STI interventions targeting young people on a larger scale.

Conclusion

Implementation science has the potential to support the delivery and dissemination of HIV/STI prevention mHealth interventions in LMICs. Notably, this review demonstrates the acceptability, appropriateness, and feasibility of mHealth interventions to promote HIV/STI prevention among young people in LMICs. However, more research is needed in this area to evaluate setting-level adoption, fidelity, widespread penetration, cost, and sustainability. Doing so will enhance the field’s understanding of the mechanisms by which these interventions lead or do not lead to changes in high HIV/STI risk and vulnerability among young people in LMICs.

Availability of data and materials

Not applicable

Abbreviations

FU:

Florida Uzoaru

HIV:

Human immunodeficiency virus

JI:

Juliet Iwelunmor

JP:

Jessica Parker

KM:

Karan Modi

LMIC:

Low- and middle-income countries

MeSH:

Medical subject headings

mHealth:

Mobile Health

MSM:

Men who have sex with men

OE:

Oliver Ezechi

PRISMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

RCT:

Randomized control trial

SM:

Stacey Mason

SMa:

Sunita Manu

STI:

Sexually transmitted infections

TS:

Thembekile Shato

UN:

Ucheoma Nwaozuru

VC:

Victoria Carter

References

  1. Mobile cellular subscriptions [https://data.worldbank.org/indicator/IT.CEL.SETS]

  2. Forrest JI, Wiens M, Kanters S, Nsanzimana S, Lester RT, Mills EJ. Mobile health applications for HIV prevention and care in Africa. Curr Opinion HIV AIDS. 2015;10(6):464–71.

    Article  Google Scholar 

  3. ITU releases 2018 global and regional ICT estimates [https://www.itu.int/en/mediacentre/Pages/2018-PR40.aspx]

  4. Mechael PN. The case for mHealth in developing countries. Innovations Technol Governance Globalization. 2009;4(1):103–18.

    Article  Google Scholar 

  5. Sloninsky D, Mechael P. Towards the development of an mhealth strategy: a literary review. New York: World Health Organization and Earth Institute; 2008.

    Google Scholar 

  6. Consulting VW. mHealth for development: the opportunity of mobile technology for healthcare in the developing world. UK: Washington Dc and Berkshire; 2009.

    Google Scholar 

  7. Kahn JG, Yang JS. Kahn JS: ‘Mobile’health needs and opportunities in developing countries. Health Affairs. 2010;29(2):252–8.

    Article  PubMed  Google Scholar 

  8. Akter S, Ray P. mHealth-an ultimate platform to serve the unserved. Yearbook Med Informatics. 2010;19(01):94–100.

    Article  Google Scholar 

  9. Higgs ES, Goldberg AB, Labrique AB, Cook SH, Schmid C, Cole CF, Obregón RA. Understanding the role of mHealth and other media interventions for behavior change to enhance child survival and development in low-and middle-income countries: an evidence review. J Health Commun. 2014;19(sup1):164–89.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Kalamar AM, Bayer AM, Hindin MJ. Interventions to prevent sexually transmitted infections, including HIV, among young people in low-and middle-income countries: a systematic review of the published and gray literature. J Adolescent Health. 2016;59(3):S22–31.

    Article  Google Scholar 

  11. Young SD, Cumberland WG, Nianogo R, Menacho LA, Galea JT, Coates T. The HOPE social media intervention for global HIV prevention in Peru: a cluster randomised controlled trial. Lancet HIV. 2015;2(1):e27–32.

    Article  PubMed  Google Scholar 

  12. Zhao Y, Zhu X, Pérez AE, Zhang W, Shi A, Zhang Z, et al. MHealth approach to promote Oral HIV self-testing among men who have sex with men in China: a qualitative description. BMC Public Health. 2018;18(1):1146.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Cole-Lewis H, Kershaw T. Text messaging as a tool for behavior change in disease prevention and management. Epidemiol Reviews. 2010;32(1):56–69.

    Article  Google Scholar 

  14. Rokicki S, Fink G. Assessing the reach and effectiveness of mHealth: evidence from a reproductive health program for adolescent girls in Ghana. BMC Public Health. 2017;17(1):969.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Rokicki S, Cohen J, Salomon JA, Fink G. Impact of a text-messaging program on adolescent reproductive health: a cluster–randomized trial in Ghana. Am J Public Health. 2017;107(2):298–305.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Ippoliti NB, L’Engle K. Meet us on the phone: mobile phone programs for adolescent sexual and reproductive health in low-to-middle income countries. Reproductive Health. 2017;14(1):11.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Merrill J, Hershow R, Gannett K, Barkley C: Pretesting an mHealth intervention for at-risk adolescent girls in Soweto, South Africa: studying the additive effects of SMSs on improving sexual reproductive health & rights outcomes. In: Proceedings of the Sixth International Conference on Information and Communications Technologies and Development: Notes-Volume 2: 2013: ACM; 2013: 96-99.

  18. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research. 2011;38(2):65–76.

    Article  PubMed  Google Scholar 

  19. Görlach MG, Schrage T, Bokemeyer C, Kröger N, Müller V, Petersen C, et al. Implementation analysis of patient reported outcomes (PROs) in oncological routine care: an observational study protocol. Health Quality Life Outcomes. 2020;18(1):1–7.

    Article  Google Scholar 

  20. Catalani C, Philbrick W, Fraser H, Mechael P, Israelski DM. mHealth for HIV treatment & prevention: a systematic review of the literature. The open AIDS J. 2013;7:17.

    Article  PubMed  Google Scholar 

  21. Moher D, Liberati A, Tetzlaff J, Altman D. Group TP, Oxman A, Cook D, Guyatt G, Swingler G, Volmink J, Ioannidis J, Young C, Horton R, et al. Preferred Reporting Items for Systematic Reviews and Meta-Analyses. 2009;6:e1000097.

    Google Scholar 

  22. Bank W: World Bank country and lending groups. In.: World Bank Data Help Desk Washington (DC); 2017.

  23. Widman L, Nesi J, Kamke K, Choukas-Bradley S, Stewart J. Technology-based interventions to reduce sexually transmitted infections and unintended pregnancy among youth. J Adolescent Health. 2018;62(6):651–60.

    Article  Google Scholar 

  24. L’Engle KL, Mangone ER, Parcesepe AM, Agarwal S, Ippoliti NB. Mobile phone interventions for adolescent sexual and reproductive health: a systematic review. Pediatrics. 2016;138(3):e20160884.

    Article  PubMed  Google Scholar 

  25. Ihesie CA. Is mobile health (mHealth) the magic bullet? A short review of the impact of mHealth on adolescent sexual health. J Public Health Epidemiol. 2015;7(8):258–62.

    Article  Google Scholar 

  26. Chigona W, Nyemba M, Metfula A: A review on mHealth research in developing countries. J Community Informatics 2012, 9(2).

  27. Muessig KE, Nekkanti M, Bauermeister J, Bull S, Hightow-Weidman LB. A systematic review of recent smartphone, Internet and Web 2.0 interventions to address the HIV continuum of care. Current HIV/AIDS Reports. 2015;12(1):173–90.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Burns K, Keating P, Free C. A systematic review of randomised control trials of sexual health interventions delivered by mobile technologies. BMC Public Health. 2016;16(1):778.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Ugalde A, Gaskin CJ, Rankin NM, Schofield P, Boltong A, Aranda S, et al. A systematic review of cancer caregiver interventions: appraising the potential for implementation of evidence into practice. Psycho-Oncology. 2019;28(4):687–701.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Popay J, Roberts H, Sowden A, Petticrew M, Arai L, Rodgers M, et al. Guidance on the conduct of narrative synthesis in systematic reviews. Product ESRC Methods Programme Version. 2006;1:b92.

    Google Scholar 

  31. Higgins JP, Green S: Cochrane handbook for systematic reviews of interventions, vol. 4: Wiley; 2011.

  32. Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. Bmj. 2011;343:d5928.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Winskell K, Sabben G, Akelo V, Ondeng'e K, Obong'o C, Stephenson R, et al. A smartphone game-based intervention (Tumaini) to prevent HIV among young Africans: pilot randomized controlled trial. JMIR mHealth and uHealth. 2018;6(8):e10482.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Odeny TA, Bailey RC, Bukusi EA, Simoni JM, Tapia KA, Yuhas K, Holmes KK, McClelland RS: Effect of text messaging to deter early resumption of sexual activity after male circumcision for HIV prevention: a randomized controlled trial. J Acquired Immune Deficiency Syndromes (1999) 2014, 65(2):e50.

  35. Ybarra ML, Bull SS, Prescott TL, Birungi R. Acceptability and feasibility of CyberSenga: an Internet-based HIV-prevention program for adolescents in Mbarara, Uganda. AIDS Care. 2014;26(4):441–7.

    Article  PubMed  Google Scholar 

  36. C-h L. Zhao Q, Gao E-S, Shah IH: Can the Internet be used effectively to provide sex education to young people in China? J Adolesc Health. 2006;39(5):720–8.

    Article  Google Scholar 

  37. Zhu X, Zhang W, Operario D, Zhao Y, Shi A, Zhang Z, et al. Effects of a mobile health intervention to promote HIV self-testing with MSM in China: a randomized controlled trial. AIDS Behavior. 2019;23(11):3129–39.

    Article  PubMed  Google Scholar 

  38. Odeny TA, Bailey RC, Bukusi EA, Simoni JM, Tapia KA, Yuhas K, et al. Text messaging to improve attendance at post-operative clinic visits after adult male circumcision for HIV prevention: a randomized controlled trial. PLoS ONE. 2012;7(9):e43832.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Ybarra M, Biringi R, Prescott T, Bull SS. Usability and navigability of an HIV/AIDS internet intervention for adolescents in a resource limited setting. Comput Informatics Nurs. 2012;30(11):587.

    Article  Google Scholar 

  40. Ybarra ML, Bull SS, Prescott TL, Korchmaros JD, Bangsberg DR, Kiwanuka JP. Adolescent abstinence and unprotected sex in CyberSenga, an Internet-based HIV prevention program: randomized clinical trial of efficacy. PLoS One. 2013;8(8):e70083.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  41. Verhey IJ, Ryan GK, Scherer N, Magidson JF. Implementation outcomes of cognitive behavioural therapy delivered by non-specialists for common mental disorders and substance-use disorders in low-and middle-income countries: a systematic review. Int J Mental Health Syst. 2020;14(1):1–14.

    Article  Google Scholar 

  42. Kemp CG, Jarrett BA, Kwon C-S, Song L, Jetté N, Sapag JC, et al. Implementation science and stigma reduction interventions in low-and middle-income countries: a systematic review. BMC Med. 2019;17(1):6.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Kemp CG, Weiner BJ, Sherr KH, Kupfer LE, Cherutich PK, Wilson D, et al. Implementation science for integration of HIV and non-communicable disease services in sub-Saharan Africa: a systematic review. Aids. 2018;32:S93–S105.

    Article  PubMed  Google Scholar 

  44. Sabben G, Mudhune V, Ondeng'e K, Odero I, Ndivo R, Akelo V, et al. A Smartphone Game to Prevent HIV Among Young Africans (Tumaini): assessing intervention and study acceptability among adolescents and their parents in a randomized controlled trial. JMIR mHealth uHealth. 2019;7(5):e13049.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Gardner W. Native VOICES: Adapting a video-based sexual health intervention for American Indian teens and young adults using the ADAPT-ITT Model. Am Indian Alaska Native Mental Health Research. 2016;23(1):24.

    Article  Google Scholar 

  46. Kerrigan D, Moreno L, Rosario S, Sweat M. Adapting the Thai 100% condom programme: developing a culturally appropriate model for the Dominican Republic. Culture Health Sexuality. 2001;3(2):221–40.

    Article  Google Scholar 

  47. Lauricella M, Valdez JK, Okamoto SK, Helm S, Zaremba C. Culturally grounded prevention for minority youth populations: a systematic review of the literature. J Primary Prevention. 2016;37(1):11–32.

    Article  Google Scholar 

  48. Airhihenbuwa CO, Ford CL, Iwelunmor JI. Why culture matters in health interventions: lessons from HIV/AIDS stigma and NCDs. Health Educ Behavior. 2014;41(1):78–84.

    Article  Google Scholar 

  49. Muula AS, Mfutso-Bengo JM. Important but neglected ethical and cultural considerations in the fight against HIV/AIDS in Malawi. Nurs Ethics. 2004;11(5):479–88.

    Article  PubMed  Google Scholar 

  50. Maticka-Tyndale E. Condoms in sub-Saharan Africa. Sexual Health. 2012;9(1):59–72.

    Article  PubMed  Google Scholar 

  51. Green LW, Glasgow RE. Evaluating the relevance, generalization, and applicability of research: issues in external validation and translation methodology. Eval Health Professions. 2006;29(1):126–53.

    Article  Google Scholar 

  52. Castro FG, Barrera M, Martinez CR. The cultural adaptation of prevention interventions: resolving tensions between fidelity and fit. Prevention Science. 2004;5(1):41–5.

    Article  PubMed  Google Scholar 

  53. Lewis CC, Klasnja P, Powell BJ, Lyon AR, Tuzzio L, Jones S, et al. From classification to causality: advancing understanding of mechanisms of change in implementation science. Front Public Health. 2018;6:136.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Lyon AR, Koerner K. User-centered design for psychosocial intervention development and implementation. Clin Psychol Sci Pract. 2016;23(2):180–200.

    Article  Google Scholar 

  55. Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual framework for implementation fidelity. Implementation Sci. 2007;2(1):40.

    Article  Google Scholar 

  56. Onakomaiya D, Gyamfi J, Iwelunmor J, Opeyemi J, Oluwasanmi M, Obiezu-Umeh C, et al. Implementation of clean cookstove interventions and its effects on blood pressure in low-income and middle-income countries: systematic review. BMJ Open. 2019;9(5):e026517.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Keith RE, Hopp FP, Subramanian U, Wiitala W, Lowery JC. Fidelity of implementation: development and testing of a measure. Implementation Science. 2010;5(1):1–11.

    Article  Google Scholar 

  58. Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychology. 2008;41(3-4):327–50.

    Article  Google Scholar 

  59. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Science. 2009;4(1):50.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Wozney L, McGrath PJ, Gehring ND, Bennett K, Huguet A, Hartling L, et al. eMental healthcare technologies for anxiety and depression in childhood and adolescence: systematic review of studies reporting implementation outcomes. JMIR Mental Health. 2018;5(2):e48.

    Article  PubMed  PubMed Central  Google Scholar 

  61. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Administration Policy Mental Health Mental Health Serv Res. 2009;36(1):24–34.

    Article  Google Scholar 

  62. Del Boca FK, McRee B, Vendetti J, Damon D. The SBIRT program matrix: a conceptual framework for program implementation and evaluation. Addiction. 2017;112:12–22.

    Article  PubMed  Google Scholar 

  63. Kemp CG, Velloza J. Implementation of eHealth interventions across the HIV care cascade: a review of recent research. Current HIV/AIDS Reports. 2018;15(6):403–13.

    Article  PubMed  PubMed Central  Google Scholar 

  64. Stephani V, Opoku D, Quentin W. A systematic review of randomized controlled trials of mHealth interventions against non-communicable diseases in developing countries. BMC Public Health. 2016;16(1):572.

    Article  PubMed  PubMed Central  Google Scholar 

  65. Louie E, Barrett EL, Baillie A, Haber P, Morley KC. Implementation of evidence-based practice for alcohol and substance use disorders: protocol for systematic review. Systematic Reviews. 2020;9(1):1–6.

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank Dr. Donghua Tao for her assistance with literature search. We would also like to thank the PATC3H consortium funded by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) Grant number: UG3HD096929

Funding

Not applicable

Author information

Authors and Affiliations

  1. College for Public Health and Social Justice, Saint Louis University, Salus center, 3545 Lafayette Avenue, Saint Louis, Missouri, 63104, USA

    Ucheoma Nwaozuru, Chisom Obiezu-Umeh, Florida Uzoaru, Stacey Mason, Victoria Carter, Sunita Manu, Karan Modi, Jessica Parker & Juliet Iwelunmor

  2. Brown School, Washington University in Saint Louis, 1 Brookings Drive, Saint Louis, Missouri, 63130, USA

    Thembekile Shato

  3. Nigerian Institute of Medical Research, 6 Edmund Crescent, Yaba, Lagos, Nigeria

    Oliver Ezechi

Authors
  1. Ucheoma Nwaozuru
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chisom Obiezu-Umeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thembekile Shato
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Florida Uzoaru
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Stacey Mason
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Victoria Carter
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Sunita Manu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Karan Modi
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Jessica Parker
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Oliver Ezechi
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Juliet Iwelunmor
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

UN designed the study, with inputs from JI and CO. UN and CO independently reviewed titles/abstracts and extracted data, in consultation with TS in case of conflicts. UN wrote the first draft of the manuscript. All authors (CO, TS, FU, KM, SM, VC, OE, JP, SMa, JI) reviewed drafts for critical content and provided permission for submission for publication. The authors read and approved the final manuscript.

Corresponding author

Correspondence to Ucheoma Nwaozuru.

Ethics declarations

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nwaozuru, U., Obiezu-Umeh, C., Shato, T. et al. Mobile health interventions for HIV/STI prevention among youth in low- and middle-income countries (LMICs): a systematic review of studies reporting implementation outcomes. Implement Sci Commun 2, 126 (2021). https://doi.org/10.1186/s43058-021-00230-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s43058-021-00230-w

Keywords

Implementation Science Communications

ISSN: 2662-2211