Skip to main content
Download PDF
Download ePub

Validity and reliability of the transculturally adapted Spanish version of the Implementation Leadership Scale (ILS)

Implementation Science Communications volume 4, Article number: 112 (2023) Cite this article

Abstract

Background

There is a need for pragmatic and reliable measures of sound factors that affect evidence-based practice (EBP) adoption and implementation in different languages and cultural environments. The Implementation Leadership Scale (ILS) is a brief and efficient measurement tool of strategic leadership for EBP implementation. The objective of this study was to assess the psychometric properties of the Spanish version of the ILS.

Methods

The process of translation of the original ILS into Spanish consisted of forward translation, panel meeting, and back-translation. Scale face and content validity compared to that of the original version were assessed and ensured before agreement on the final version. Psychometric properties were examined in 144 healthcare professionals (family physicians, pediatricians, practice and pediatric nurses) involved in implementation or improvement research projects. ILS factor structure was tested by confirmatory factor analysis (CFA). Reliability was assessed by internal consistency analysis. The Pearson correlation between the ILS and the Organizational Support dimension of the Organizational Readiness for Knowledge Translation (OR4KT) questionnaire in the subsample of pediatricians and pediatric nurses (n = 52) was estimated for convergent validity analysis.

Results

The CFA results indicated that the original four theorized first-order factors with a second-order Implementation Leadership factor fit the data well (χ2 = 107.70; df = 45; p < 0.001). All standardized first- and second-order factor loadings were statistically significant. Fit indexes showed acceptable figures (GFI = 0.90; CFI = 0.97; RMSEA = 0.10; SRMR = 0.053). Cronbach’s alpha coefficient for the four dimensions of ILS ranged from 0.90 to 0.97, while the reliability estimated for the total scale was 0.95. Results of convergent validity revealed high correlation (r = 0.56) between the ILS and the OR4KT’s Organizational Support dimension.

Conclusion

The CFA results demonstrated that the tested first- and second-order factor structure of the 12-item Spanish version of the ILS is consistent with the factor structure of the original tool. The availability of the ILS will allow Spanish-speaking researchers to assess and advance understanding of the implementation leadership construct as a predictor of organizational implementation context.

Peer Review reports

Background

Implementation research is defined as the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices (EBPs) into routine practice, and, hence, to improve the quality and effectiveness of health services [1]. It includes the study of influences on healthcare professionals and organizational behavior [1]. Factors influencing EBP implementation success include individual provider factors as well as organizational factors (i.e., organizational culture, climate, and leadership) that are expected to have an even more profound effect on EBP integration [2]. Of these factors, leadership has been identified as one of the most significant components of the organizational context that may act as a mechanism for improving EBP implementation [3, 4] and therefore impact on organizational change and innovation within healthcare organizations [5].

Leadership at several organizational levels, from top to first-level leaders, facilitates relevant mechanisms in promoting implementation, such as positive organizational climate in the workplace [6], positive employee attitudes towards work [7] and adopting EBPs [8], positive trusting inter-professional relationships and supportive collaborative working vision [9], and commitment to organizational change [10]. Despite the critical role of first-level leaders, those who directly supervise and manage direct service providers, in the translation of clinical research and practice change, little is known about how to identify and measure leadership behaviors in health professionals [11]. In this regard, Aarons et al. developed the Implementation Leadership Scale (ILS) in order to assess the degree to which leaders engage in specific leadership behaviors that are critical for effective EBP implementation [12]. The scale has four dimensions: (1) proactive leadership, which addresses the degree to which a leader establishes clear goals and plans as well as removes obstacles to EBP implementation; (2) knowledgeable leadership, which measures the degree to which a leader has a deep understanding of EBP and is able to answer staff’s EBP implementation-related questions; (3) supportive leadership, which represents the degree of the leader’s support and recognition of staff’s efforts to learn, use and adopt EBPs; and (4) perseverant leadership, which reflects the degree to which a leader is consistent, unwavering and responsive to the ups and downs, challenges and issues that arise in the EBP implementation process [12].

The ILS was originally developed and tested in mental health settings [12] and has then been cross-validated and used in substance use facilities [13], child welfare service organizations [14], education sectors [15], and acute care in nursing contexts [16], as well as with mental health clinic supervisors [17]. Nonetheless, to the best of our knowledge, so far, the ILS has been only translated into Chinese [18, 19], Greek [20], Norwegian [21], and German [22]. Hence, we consider that there is a need for pragmatic and reliable measures of sound factors that affect EBP adoption and implementation in different cultural environments and languages, including Spanish.

In this context, the main objective of this study was to assess the psychometric properties of a Spanish version of the Implementation Leadership Scale, in order to evaluate whether this version measures in the same way and with the same assurance as the original English version.

Methods

Participants and procedure

Participants were family physicians, pediatricians, practice nurses, and pediatric nurses, who were involved in implementation or improvement research projects conducted across the 13 Integrated Healthcare Organizations (IHOs) of Osakidetza – Basque Health Service (References: PI15/00350 and PI19/00234). These projects involved an implementation strategy that included leadership promotion aimed at fostering both the knowledge and skills regarding the EBP of interest as well as the capacity to lead the EBP implementation process in order to promote the EBP’s translation and integration into routine practice, of one of the practice healthcare professionals who was elected as leader by consensus within the collaborating centers. The present sample responds to a convenience sampling method as it consisted of professionals who responded to several procedures via online surveys used to evaluate the implementation or improvement projects in which they were involved. All these online surveys provided information about the main objectives of the study, data confidentiality, and the planned use of the resulting data. All participants voluntarily provided their written informed consent before taking part in the study. The studies within which these surveys were planned and conducted were reviewed and approved by the Basque Country Clinical Research Ethics Committee (CEIm Euskadi) (References: 08/2015 and 07/2019).

Osakidetza—Basque Health Service provides universal coverage and services are free at the point of use, aside from co-payment for drugs, funded through regional general taxation. Primary, specialized, and social health-related service provision is organized around 13 IHOs, with 135 primary care health centers and 9 hospitals, that cover the 3 provinces of the region of the Basque Country: Araba, Bizkaia, and Gipuzkoa. Each resident is on the list of one family physicians or pediatrician who offers comprehensive primary care and refers patients for hospital and specialty services. Services users can also attend emergency care by a point of care service provided by hospitals.

Instruments

Implementation leadership scale (ILS)

The ILS consists of four factors: 1) Proactive leadership, 2) Knowledgeable leadership, 3) Supportive leadership, and 4) Perseverant leadership [12]. Each factor is assessed with three items, which are scored on a 0 (“not at all”) to 4 (“to a very great extent”) scale. The score obtained indicates the degree to which a leader/supervisor is engaged in the abovementioned behaviors with regard to EBP implementation. There are two versions of the ILS, one for staff to report about leader/supervisor, and another for leaders/supervisors to report about themselves. The staff version of the ILS was used in this study. The score for each subscale is created by computing a mean score for each set of items on a given subscale. The mean score for the total ILS is obtained from the mean of the four subscale scores.

Organizational readiness for knowledge translation (OR4KT) questionnaire

The Organizational Support dimension of the Spanish version of the OR4KT questionnaire was used for the assessment of convergent validity [23, 24]. This questionnaire was developed as a tool for assessing healthcare organizational readiness to change clinical practice in order to enable EBP implementation and adoption of proven interventions. The OR4KT questionnaire was originally developed in English and then translated into Spanish and French [24]. The questionnaire consists of 59 items that assess 6 dimensions and 23 sub-dimensions related to organizational predisposition to knowledge translation: organizational climate, organizational support, contextual factors, change content, leadership, and motivation. Organizational support includes four sub-dimensions: support climate (items 40 to 43), monitoring (items 44 and 45), evaluation process (items 46 to 48), and feedback (item 49). Each item is scored on a five-point Likert scale from 1 (“strongly disagree”) to 5 (“strongly agree”). The total score is computed by summing the scores on each item, with a maximum score of 295 points. Then, this score is normalized on a 0 to 100 scale to ease interpretation.

Transcultural adaptation procedure, and face and content validity assessment

The ILS original version was translated into Spanish by a translation/back-translation procedure [25]. The process consisted of the following steps: forward translation, panel meeting, back-translation, assessment of face and content validity and agreement on a final version.

In the first step, the original ILS was translated into Spanish by experienced native Spanish-speaker translators who were members of the research team (AS, PM). Special attention was paid to the readability and clarity of the items. In the second step, the translated Spanish version was reviewed by bilingual members of the research team including the translators (GG, SP, AS, PM) seeking to detect any inadequate expressions/concepts of the translation and discrepancies between the forward translation and the existing English version. In the third step, an independent native English speaker (HR), who was fluent in Spanish and had no knowledge of the original ILS, performed the back-translation. The original English version and the back-translation were compared and the conceptual and cultural equivalence of the transculturally-adapted version was assessed, placing special emphasis on the linguistic equivalence. There were only small discrepancies which were discussed until consensus was reached concerning minor modifications required.

Finally, a group of researchers from the Primary Care Research Unit of Bizkaia, who were not involved in the present study or in the implementation or improvement projects within which we sought to validate the Spanish ILS, evaluated the extent to which the translated version maintained the face and content validity of the original version and assessed the clarity of the wording of the items. Minor final corrections were made based on researchers’ comments. The final Spanish version of the ILS instrument used in the validation process is presented in Additional file 1.

Statistical analysis

Confirmatory factor analysis (CFA) was conducted to confirm the ILS factor structure. In a first step, we fitted a model in which we specified which sub-dimension each of the items belonged to. In a second step, we carried out a second-order analysis, checking the fit of a model in which each sub-dimension was loaded onto its reference dimension and these, in turn, onto a global dimension that measures implementation leadership. The model fit was assessed using the value of the chi-square/degree of freedom ratio (χ2/df) and also the comparative fit index (CFI), the goodness of fit index (GFI), the root mean square error of approximation (RMSEA), and the standardized root mean square residual (SRMR). Values of χ2/df ≤ 2.5, CFI ≥ 0.95, GFI ≥ 0.90, RMSEA ≤ 0.06, and SRMR ≤ 0.08 indicate acceptable model fit [26]. Reliability was quantified in terms of consistency and Cronbach’s alpha coefficients were calculated to estimate internal consistency for each of the subscales and the total scale. A value of α ≥ 0.7 was considered good [27]. Pearson coefficients were used to assess convergent validity of the ILS total scale and four subscales with the Organizational Support dimension, and its four sub-dimensions, of the OR4KT questionnaire. The correlation is considered high when this coefficient is greater than 0.5, and moderate when the coefficient is > 0.3 [28]. SAS (v. 9.2, SAS Institute, Cary, NC, USA) and IBM SPSS (version 23.0; Armonk, NY, USA) were used to perform all the statistical analysis.

Results

Translation of the ILS tool into Spanish and back-translation into English did not encounter any major content or language issues. Each of the items were precisely and comprehensibly worded and completely captured the concept addressed by the original one, resulting in a Spanish version of the ILS conceptually equivalent to the English instrument. The psychometric properties of the Spanish ILS were examined in a sample of 144 healthcare professionals that consisted of family physicians (n = 51; 35.4%), pediatricians (n = 38; 26.4%), practice nurses (n = 48; 33.3%) and pediatric nurses (n = 7; 4.9%). Out of them, 117 professionals (81.3%) were from a total of 33 primary care health centers and 27 (18.7%) from 5 specialized care settings across 12 out of the 13 IHOs of Osakidetza – Basque Health Service. The mean age of participants was 48.34 years (SD = 9.66; range 25–65) and the majority were female (81.3%).

The descriptive and reliability statistics for the total scale and the four subscales of the ILS are reported in Table 1. The total scale mean was 3.34 (SD = 0.58), whereas means for the four subscales were between 3.20 and 3.44. Cronbach’s alpha coefficients showed very good internal consistency for the four dimensions of the ILS, with values ranging from 0.90 to 0.97, while the reliability estimated for the total scale was 0.95.

Table 1 Descriptive and reliability statistics of the Spanish version of the Implementation Leadership Scale (ILS)
Full size table

The CFA confirmed the already designed factor structure of the ILS. In this regard, the CFA results indicated that the original four theorized first-order factors with a second-order Implementation Leadership factor fit the data well (χ2 = 107.70; df = 45; p < 0.001). Moreover, fit indexes showed acceptable figures (GFI = 0.90; CFI = 0.97; RMSEA = 0.10; SRMR = 0.053). Figure 1 displays the standardized factor loadings for the higher-order factor model. First-order factor loadings ranged from 0.82 to 0.98, while second-order factor loadings ranged from 0.71 to 0.88. All standardized first- and second-order factor loadings were statistically significant (p < 0.001).

Fig. 1
figure 1

Second-order confirmatory factor analysis factor loadings for the Spanish version of the Implementation Leadership Scale (ILS). Note: n = 144; All factor loadings are standardized and are statistically significant, p < 0.001; χ.2 = 107.70, df = 45, p < 0.001; GFI = 0.90; CFI = 0.97; RMSEA = 0.10; SRMR = 0.053

Full size image

Convergent validity was assessed in the subsample of pediatricians and pediatric nurses (n = 52). As expected, convergent validity analysis revealed high correlation (Pearson coefficient, r = 0.56) between the ILS total scale and the Organizational Support dimension of the OR4KT questionnaire. In addition, the analysis between the OR4KT’s Organizational Support dimension and two of the ILS subscales (proactive and perseverant leadership) showed high correlation (r = 0.51 and r = 0.54, respectively). Moreover, ILS total scale also displayed high correlations (r = 0.52) with the sub-dimensions support climate and evaluation process of the OR4KT’s Organizational Support dimension (Table 2).

Table 2 Pearson correlations of the Spanish Implementation Leadership Scale (ILS) scores with the Organizational Support dimension of the Organizational Readiness for Knowledge Translation (OR4KT) questionnaire scores
Full size table

Discussion

This study aimed to assess the psychometric properties of the Spanish version of the Implementation Leadership Scale (ILS). The ILS is a brief and efficient measurement tool of strategic leadership for EBP implementation. The instrument consists of 12 items grouped under four dimensions assessing the degree to which a leader is proactive, knowledgeable, supportive, and perseverant in the implementation of EBP. Firstly, we developed a linguistically equivalent version in Spanish of the original English version of the ILS that was developed by Aarons et al. [12]. After having established the face and content validity of the Spanish version, we proceeded with the validation process by assessing its factor structure and psychometric properties (reliability and validity) in the context of implementation or improvement projects conducted in Osakidetza – Basque Health Service.

The results of the initial validation process indicated that the Spanish ILS is a reliable instrument based on the internal consistency assessment. The Cronbach’s alpha coefficient for the total scale is 0.95, which provides strong support for the reliability of this tool. This value is comparable to that obtained in the assessment of the original English version of the ILS (α = 0.98) [12] and in other cross-validation studies using the employee ratings (α = 0.97; 0.99 and 0.98) [13, 15, 16] or the leader self-ratings (α = 0.95) [22]. When compared to the other available translations of the ILS (i.e., the Chinese [18, 19], Greek [20], Norwegian [21], and German [22] versions), the internal consistency of the Spanish ILS is as excellent as the Norwegian ILS, with Cronbach’s alpha coefficients between 0.93 and 0.97 for the four dimensions and 0.96 for the total scale [21]. In contrast, the Chinese, Greek, and German versions of ILS showed good/adequate reliability with Cronbach’s alpha coefficients of 0.86–0.95 for the four subscales and 0.93 for total scale of the Chinese ILS [19]; 0.85–0.91 for the four subscales and 0.94 for total scale of the Greek ILS [20], and 0.875 for the total scale and of 0.77–0.92 for the four factors of the German ILS [22].

Secondly, to assess the validity of the Spanish version of the ILS, we performed CFA to provide evidence of internal structure validity. The CFA results confirmed a good model fit to the theorized first- and second-order factor structure of the Spanish ILS as previously reported in several studies in different settings, as well as showing acceptable factor loadings [12,13,14,15,16,17, 19,20,21]. Our findings demonstrated the factor structure of the Spanish version of the ILS as well as its structural validity in the context of primary care and pediatrics departments (both primary and specialized care) when completed by family physicians, pediatricians, practice nurses, and pediatric nurses.

Most ILS validation studies have used the Multifactor Leadership Questionnaire (MLQ) [29] to evaluate convergent validity, reporting that the ILS had moderate to high correlations with the MLQ total and subscales [12, 13, 19, 21]. However, other scales have been used for assessing convergent validity, as for example the Quality of leadership dimension of the Copenhagen Psychosocial Questionnaire Version II (COPSOQ II) [30] in the Greek ILS validation study (r = 0.792) [20]. In our study, we have used the Organizational Support dimension of the OR4KT questionnaire, which measures an organization’s level of readiness to implement EBPs; and is available in Spanish, having been developed, translated into Spanish, and transculturally validated by members of our research group [23, 24]. Specifically, we have focused on the Organizational Support dimension which assesses whether there is a supportive climate within a healthcare center, and the reception of monitoring, evaluation processes, and feedback, all of them reflecting the behaviors expected to be performed by the leader as part of the implementation strategy in the projects involved in the present validation study. In short, we have observed a high correlation (r = 0.56) between the Spanish ILS total scale and the Organizational Support dimension of the OR4KT questionnaire, indicating good convergent validity in our clinical setting; though this was only assessed in the subsample of pediatricians and pediatric nurses (n = 52).

Our study contributes to the literature not only by demonstrating for the first time the psychometric properties of the ILS in Spanish but also by including for the first time results from a Spanish sample of both physicians and nurses in a primary care setting (81.3% out of the total sample reported working in primary care health centers). To our knowledge, the ILS has not been validated with physicians yet [11]; and therefore our study is the first to have included physicians (61.8% out of the total sample) who rated their leaders. Our findings are in line with the results from previous studies in other languages and in different settings where the staff version of ILS was validated among clinicians working at mental health clinics in the USA [12] and Norway [21]; service providers at substance use disorder treatment agencies [13]; employees at child welfare service organizations [14]; employees in the education sector [15]; nurses and midwives in Greece [20] and nurses in the USA and China [16, 19]. Nonetheless, it should be pointed out that in this study, we validated the Spanish staff version of the ILS. Since discrepancies were previously reported between supervisor and clinician/follower ratings [31,32,33], our results would not be comparable with the assessment of the German ILS, where they validated the leader version with physicians [22], or with a previously reported cross-validation study of the ILS with mental health clinic supervisors’ self-ratings [17].

A limitation of the current study was the final sample size (n = 144), which was somewhat smaller than that in previous validation studies, which included above 200 participants [12,13,14, 16, 19, 21]. However, it should be noted that similar good results with regard to the factor structure and the internal consistency (reliability) were obtained in our study. Second, the data were collected from a sample consisting of family physicians, pediatricians, practice nurses, and pediatric nurses and not from other specialties. On the other hand, the fact that the study was conducted in 12 out of the 13 IHOs of Osakidetza – Basque Health Service and that the participants were from numerous different primary care health centers could contribute to its generalizability.

Future research should be considered in order to evaluate the properties of the Spanish version of the ILS among different healthcare professionals as well as in different regions in Spain and Spanish-speaking countries with different cultural environments. Our aim is to promote implementation research in the Spanish-speaking scientific community by the development of implementation research tools to support the design, evaluation and reporting of implementation research projects in non-English speaking context.

Conclusions

There is a strong need for pragmatic, efficient measures that assess the construct of leadership in the context of EBP implementation in different languages and cultural environments. This study led to the translation, transcultural adaptation, and assessment of face and content validity, internal structure validity, and psychometric properties of the Spanish version of the ILS. The CFA results demonstrated that the tested first- and second-order factor structure of the 12-item Spanish version of the ILS is consistent with the factor structure of the original tool. The availability of the Spanish version of this instrument will allow Spanish-speaking researchers to assess and advance understanding of the implementation leadership construct as a predictor of organizational implementation context.

Availability of data and materials

The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

CFA:

Confirmatory factor analysis

CFI:

Comparative Fit Index

EBP:

Evidence-based practice

GFI:

Goodness of Fit Index

IHO:

Integrated healthcare organization

ILS:

Implementation Leadership Scale

M:

Mean

OR4KT:

Organizational Readiness for Knowledge Translation

RMSEA:

Root mean square error of approximation

SD:

Standard deviation

SRMR:

Standardized root mean square residual

χ2/df :

Chi-square/degree of freedom ratio

References

  1. Eccles MP, Mittman BS. Welcome to Implementation Science. Implement Sci. 2006;1:1. https://doi.org/10.1186/1748-5908-1-1.

    Article  PubMed Central  Google Scholar 

  2. Aarons GA, Sommerfeld DH. Leadership, innovation climate, and attitudes toward evidence-based practice during a statewide implementation. J Am Acad Child Adolesc Psychiatry. 2012;51(4):423–31. https://doi.org/10.1016/j.jaac.2012.01.018.

    Article  PubMed  Google Scholar 

  3. Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M, et al. A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. 2013;8:35. https://doi.org/10.1186/1748-5908-8-35.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011;38(1):4–23. https://doi.org/10.1007/s10488-010-0327-7.

    Article  PubMed  Google Scholar 

  5. Aarons GA, Ehrhart MG, Farahnak LR, Sklar M. Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annu Rev Public Health. 2014;35:255–74. https://doi.org/10.1146/annurev-publhealth-032013-182447.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Green AE, Albanese BJ, Cafri G, Aarons GA. Leadership, organizational climate, and working alliance in a children’s mental health service system. Community Ment Health J. 2014;50(7):771–7. https://doi.org/10.1007/s10597-013-9668-5.

    Article  PubMed  Google Scholar 

  7. Kinjerski V, Skrypnek BJ. The promise of spirit at work: increasing job satisfaction and organizational commitment and reducing turnover and absenteeism in long-term care. J Gerontol Nurs. 2008;34(10):17–27. https://doi.org/10.3928/00989134-20081001-03.

    Article  PubMed  Google Scholar 

  8. Aarons GA. Transformational and transactional leadership: association with attitudes toward evidence-based practice. Psychiatr Serv. 2006;57(8):1162–9. https://doi.org/10.1176/ps.2006.57.8.1162.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Lau R, Stevenson F, Ong BN, Dziedzic K, Treweek S, Eldridge S, et al. Achieving change in primary care-effectiveness of strategies for improving implementation of complex interventions: systematic review of reviews. BMJ Open. 2015;5(12):e009993. https://doi.org/10.1136/bmjopen-2015-009993

  10. Hill NS, Seo M-G, Kang JH, Taylor MS. Building employee commitment to change across organizational levels: the influence of hierarchical distance and direct managers’ transformational leadership. Organ Sci. 2011;23(3):758–77. https://doi.org/10.1287/orsc.1110.0662.

    Article  Google Scholar 

  11. Carlson MA, Morris S, Day F, Dadich A, Ryan A, Fradgley EA, et al. Psychometric properties of leadership scales for health professionals: a systematic review. Implement Sci. 2021;16(1):85. https://doi.org/10.1186/s13012-021-01141-z.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Aarons GA, Ehrhart MG, Farahnak LR. The Implementation Leadership Scale (ILS): development of a brief measure of unit level implementation leadership. Implement Sci. 2014;9(1):45. https://doi.org/10.1186/1748-5908-9-45.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Aarons GA, Ehrhart MG, Torres EM, Finn NK, Roesch SC. Validation of the Implementation Leadership Scale (ILS) in substance use disorder treatment organizations. J Subst Abuse Treat. 2016;68:31–5. https://doi.org/10.1016/j.jsat.2016.05.004.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Finn NK, Torres EM, Ehrhart MG, Roesch SC, Aarons GA. Cross-Validation of the Implementation Leadership Scale (ILS) in Child Welfare Service Organizations. Child Maltreat. 2016;21(3):250–5. https://doi.org/10.1177/1077559516638768.

    Article  PubMed  Google Scholar 

  15. Lyon AR, Cook CR, Brown EC, Locke J, Davis C, Ehrhart M, et al. Assessing organizational implementation context in the education sector: confirmatory factor analysis of measures of implementation leadership, climate, and citizenship. Implement Sci. 2018;13(1):5. https://doi.org/10.1186/s13012-017-0705-6.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Shuman CJ, Ehrhart MG, Torres EM, Veliz P, Kath LM, VanAntwerp K, et al. EBP Implementation Leadership of Frontline Nurse Managers: Validation of the Implementation Leadership Scale in Acute Care. Worldviews Evid Based Nurs. 2020;17(1):82–91. https://doi.org/10.1111/wvn.12402.

    Article  PubMed  Google Scholar 

  17. Torres EM, Ehrhart MG, Beidas RS, Farahnak LR, Finn NK, Aarons GA. Validation of the Implementation Leadership Scale (ILS) with Supervisors’ Self-Ratings. Community Ment Health J. 2018;54(1):49–53. https://doi.org/10.1007/s10597-017-0114-y.

    Article  PubMed  Google Scholar 

  18. Hu J, Gifford W, Ruan H, Harrison D, Li Q, Ehrhart MG, et al. Translation and linguistic validation of the implementation leadership scale in Chinese nursing context. J Nurs Manag. 2019;27(5):1030–8. https://doi.org/10.1111/jonm.12768.

    Article  PubMed  Google Scholar 

  19. Hu J, Gifford W, Ruan H, Harrison D, Li Q, Ehrhart MG, et al. Validating the Implementation Leadership Scale in Chinese nursing context: A cross-sectional study. Nurs Open. 2021;8(6):3420–9. https://doi.org/10.1002/nop2.888.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Mandrou E, Tsounis A, Sarafis P. Validity and reliability of the Greek version of Implementation Leadership Scale (ILS). BMC Psychol. 2020;8(1):49. https://doi.org/10.1186/s40359-020-00413-5.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Braathu N, Laukvik EH, Egeland KM, Skar AS. Validation of the Norwegian versions of the Implementation Leadership Scale (ILS) and Multifactor Leadership Questionnaire (MLQ) in a mental health care setting. BMC Psychol. 2022;10(1):25. https://doi.org/10.1186/s40359-022-00725-8.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Söling S, Pfaff H, Karbach U, Ansmann L, Köberlein-Neu J; AdAM-Study Group. How is leadership behavior associated with organization-related variables? Translation and psychometric evaluation of the implementation leadership scale in German primary healthcare. BMC Health Serv Res. 2022;22(1):1065. https://doi.org/10.1186/s12913-022-08434-z

  23. Grandes G, Bully P, Martinez C, Gagnon MP. Validity and reliability of the Spanish version of the Organizational Readiness for Knowledge Translation (OR4KT) questionnaire. Implement Sci. 2017;12(1):128. https://doi.org/10.1186/s13012-017-0664-y.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Gagnon MP, Attieh R, Dunn S, Grandes G, Bully P, Estabrooks CA, et al. Development and Content Validation of a Transcultural Instrument to Assess Organizational Readiness for Knowledge Translation in Healthcare Organizations: The OR4KT. Int J Health Policy Manag. 2018;7(9):791–797. https://doi.org/10.15171/ijhpm.2018.17

  25. Maneesriwongul W, Dixon JK. Instrument translation process: a methods review. J Adv Nurs. 2004;48(2):175–86. https://doi.org/10.1111/j.1365-2648.2004.03185.x.

    Article  PubMed  Google Scholar 

  26. Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model. 1999;6:1–55. https://doi.org/10.1080/10705519909540118.

    Article  Google Scholar 

  27. Streiner DL, Norman GR. Health measurement scales: A practical guide to their development and use. 4th ed. Oxford: Oxford University Press; 2008.

    Book  Google Scholar 

  28. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. New York: Lawrence Earlbaum Associates; 1988.

    Google Scholar 

  29. Bass BM, Avolio BJ. MLQ: Multifactor Leadership Questionnaire. Redwood City: Mind Garden; 1995.

    Google Scholar 

  30. Pejtersen JH, Kristensen TS, Borg V, Bjorner JB. The second version of the Copenhagen Psychosocial Questionnaire. Scand J Public Health. 2010;38(3 Suppl):8–24. https://doi.org/10.1177/1403494809349858.

    Article  PubMed  Google Scholar 

  31. Aarons GA, Ehrhart MG, Farahnak LR, Finn N. Implementation leadership: Confirmatory factor analysis and supervisor-clinician discrepancy in ratings on the Implementation Leadership Scale (ILS). Implement Sci. 2015;10(Suppl 1):A70. https://doi.org/10.1186/1748-5908-10-S1-A70.

    Article  PubMed Central  Google Scholar 

  32. Aarons GA, Ehrhart MG, Torres EM, Finn NK, Beidas RS. The humble leader: association of discrepancies in leader and follower ratings of implementation leadership with organizational climate in mental health. Psychiatr Serv. 2017;68(2):115–22. https://doi.org/10.1176/appi.ps.201600062.

    Article  PubMed  Google Scholar 

  33. Aarons GA, Ehrhart MG, Farahnak LR, Sklar M, Horowitz J. Discrepancies in Leader and Follower Ratings of Transformational Leadership: Relationship with Organizational Culture in Mental Health. Adm Policy Ment Health. 2017;44(4):480–91.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors would like to thank all the healthcare professionals who volunteered to take part in the study.

Footnotes

Not applicable.

Funding

This study has been funded by Instituto de Salud Carlos III (ISCIII), co-funded by the European Union (European Regional Development Fund “A way to make Europe”), through the projects: PI15/00350, PI18/00952, PI19/00234, RD16/0007/0002, and co-funded by the European Union – NextGenerationEU funds, that finance the actions of the Recovery and Resilience Facility (Mecanismo para la Recuperación y la Resiliencia -MRR), through the project RD21/0016/0003.

Author information

Authors and Affiliations

  1. Biocruces Bizkaia Health Research Institute, Network for Research on Chronicity, Primary Care, and Health Promotion (RICAPPS), Plaza Cruces s/n, 48903, Bizkaia, Barakaldo, Spain

    Marta Llarena & Marta Gonzalez- Saenz de Tejada

  2. Biocruces Bizkaia Health Research Institute, Plaza Cruces s/n, E-48903, Bizkaia, Barakaldo, Spain

    Heather Lynn Rogers

  3. Ikerbasque Basque Foundation for Science, Bilbao, Spain

    Heather Lynn Rogers

  4. Biocruces Bizkaia Health Research Institute, Plaza Cruces s/n, 48903, Bizkaia, Barakaldo, Spain

    Patricia Macia, Mikel Rueda & Borja Santos

  5. Primary Care Research Unit of Bizkaia, Deputy Directorate of Healthcare Assistance, Biocruces Bizkaia Health Research Institute, Basque Healthcare Service – Osakidetza, Plaza Cruces s/n, 48903, Bizkaia, Barakaldo, Spain

    Susana Pablo

  6. Rontegi-Barakaldo Primary Care Center, University of the Basque Country, UPV/EHU, Biocruces Bizkaia Health Research Institute, Network for Research on Chronicity, Primary Care, and Health Promotion (RICAPPS), Bizkaia, Barakaldo, Spain

    Marta Montejo

  7. Pediatric Emergency Department, Cruces University Hospital, Biocruces Bizkaia Health Research Institute, Network for Research on Chronicity, Primary Care, and Health Promotion (RICAPPS), Plaza Cruces s/n, 48903, Bizkaia, Barakaldo, Spain

    Natalia Paniagua

  8. Pediatric Emergency Department, Cruces University Hospital, Biocruces Bizkaia Health Research Institute, University of the Basque Country, UPV/EHU, Network for Research on Chronicity, Primary Care, and Health Promotion (RICAPPS), Plaza Cruces s/n, 48903, Bizkaia, Barakaldo, Spain

    Javier Benito

  9. Primary Care Research Unit of Bizkaia, Deputy Directorate of Healthcare Assistance, Biocruces Bizkaia Health Research Institute, Basque Healthcare Service – Osakidetza, Network for Research on Chronicity, Primary Care, and Health Promotion (RICAPPS), Plaza Cruces s/n, 48903, Bizkaia, Barakaldo, Spain

    Gonzalo Grandes & Alvaro Sanchez

Authors
  1. Marta Llarena
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Heather Lynn Rogers
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Patricia Macia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Susana Pablo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marta Gonzalez- Saenz de Tejada
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Marta Montejo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Natalia Paniagua
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Javier Benito
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Mikel Rueda
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Borja Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Gonzalo Grandes
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Alvaro Sanchez
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AS conceived the study and was responsible for the design of the study. AS, MM, NP and JB were the Principal Investigators or Co-Investigators of the implementation or improvement research projects conducted in Osakidetza – Basque Health Service. AS, PM, SP, GG and HR contributed to the translation of the ILS scale. PM and MR were responsible for data collection and processing. AS designed and performed the statistical analysis. BS participated in the statistical analysis. AS and ML developed the manuscript outline and drafted the manuscript. AS, ML and MG were involved in editing and reviewing the manuscript. All authors read and approved the final version of the manuscript.

Corresponding author

Correspondence to Alvaro Sanchez.

Ethics declarations

Ethics approval and consent to participate

The Basque Country Clinical Research Ethics Committee (CEIm Euskadi) approved the research projects within which our study was conducted (References: 08/2015 and 07/2019). All professionals were properly informed about the purpose of the study and provided their written informed consent before taking part in the study. Confidentiality was maintained by data coding, to avoid the identification of data with personal information.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1. 

Spanish version of the Implementation Leadership Scale (ILS) – Staff version (PDF format).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Llarena, M., Rogers, H.L., Macia, P. et al. Validity and reliability of the transculturally adapted Spanish version of the Implementation Leadership Scale (ILS). Implement Sci Commun 4, 112 (2023). https://doi.org/10.1186/s43058-023-00495-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s43058-023-00495-3

Keywords

Implementation Science Communications

ISSN: 2662-2211