Skip to main content

Table 1 Procedure of IDAS development and validation

From: Implementation Degree Assessment Sheet for Health Program in Japan by Customizing CFIR: Development and Validation

Phase Step Method Purpose/adoption criteria
Phase 1: Item development Step 1: Identification of domain and item Choose an existing framework that has been validated. Because CFIR (five domains, 38 constructs) is a meta-theoretical framework that integrates about 20 theories and models and has already been used and validated in many studies (over 2600 citations).
Step 2: Content validity Forward translation and reconciliation: 4 individuals translated, consulted, and integrated.
Modify the contents to fit the context of Japanese health programs: 7 experts consulted.
To ensure the quality of the translation by adhering to the forward translation procedure by multiple people.
To ensure the quality of item selection and regeneration by Japanese experts with experience of health program implementation.
Phase 2: Scale development Step 3: Pre-testing Pre-test with the target group To ensure the questions and answers are meaningful.
Adopt those with at least 70% agreement for each question of "easy to understand" and "think it is important”.
Step 4: Survey administration and sample size National survey of the target population To ensure the quality of reliability and validity by calculating a sufficient sample size for the survey and distributing it evenly throughout the country.
Step 5: Item analysis Ceiling/floor effect To examine whether there is a scale attenuation effect depending on the degree of asymmetrically of the frequency distribution.
Items with mean + 1SD < 5 or mean − 1SD > 0 are deleted.
Item-total correlation analysis To check if any item in the set of tests is inconsistent with the averaged behavior of the others, and thus can be discarded.
Item-total correlation coefficient ≥ 0.5
Phase 3: Scale evaluation Step 6: Tests of reliability Cronbach’s coefficient alpha To measure the internal consistency of IDAS.
Adopted at 0.8 or higher.
Split-half correlations (odd/even) method To check the verification of stability of IDAS.
Adopted at the Spearman–Brown reliability coefficient 0.8 or higher.
Step 7: Tests of validity Construct validity To verify the model fitness of construct validity with a confirmatory factor analysis.
Check the goodness-of-fit index for fit in the same five domains as CFIR
GFI, CFI ≥ 0.9 RMSEA < 0.08
Criterion validity To estimate the extent to which a test correlates with an established standard of comparison (RUC).
Correlation coefficient ≥ 0.5
Differentiation by “Known Groups” To examine if the concept measured behaves as expected in relation to “known groups”. (Four groups depending on the years of experience and whether the respondent was a supervising PHN or not)
Significant difference between the groups (P < 0.05)