|Phase 1: Item development||Step 1: Identification of domain and item||Choose an existing framework that has been validated.||Because CFIR (five domains, 38 constructs) is a meta-theoretical framework that integrates about 20 theories and models and has already been used and validated in many studies (over 2600 citations).|
|Step 2: Content validity||
Forward translation and reconciliation: 4 individuals translated, consulted, and integrated.|
Modify the contents to fit the context of Japanese health programs: 7 experts consulted.
To ensure the quality of the translation by adhering to the forward translation procedure by multiple people.|
To ensure the quality of item selection and regeneration by Japanese experts with experience of health program implementation.
|Phase 2: Scale development||Step 3: Pre-testing||Pre-test with the target group||
To ensure the questions and answers are meaningful.|
Adopt those with at least 70% agreement for each question of "easy to understand" and "think it is important”.
|Step 4: Survey administration and sample size||National survey of the target population||To ensure the quality of reliability and validity by calculating a sufficient sample size for the survey and distributing it evenly throughout the country.|
|Step 5: Item analysis||Ceiling/floor effect||
To examine whether there is a scale attenuation effect depending on the degree of asymmetrically of the frequency distribution.|
Items with mean + 1SD < 5 or mean − 1SD > 0 are deleted.
|Item-total correlation analysis||
To check if any item in the set of tests is inconsistent with the averaged behavior of the others, and thus can be discarded.|
Item-total correlation coefficient ≥ 0.5
|Phase 3: Scale evaluation||Step 6: Tests of reliability||Cronbach’s coefficient alpha||
To measure the internal consistency of IDAS.|
Adopted at 0.8 or higher.
|Split-half correlations (odd/even) method||
To check the verification of stability of IDAS.|
Adopted at the Spearman–Brown reliability coefficient 0.8 or higher.
|Step 7: Tests of validity||Construct validity||
To verify the model fitness of construct validity with a confirmatory factor analysis.|
Check the goodness-of-fit index for fit in the same five domains as CFIR
GFI, CFI ≥ 0.9 RMSEA < 0.08
To estimate the extent to which a test correlates with an established standard of comparison (RUC).|
Correlation coefficient ≥ 0.5
|Differentiation by “Known Groups”||
To examine if the concept measured behaves as expected in relation to “known groups”. (Four groups depending on the years of experience and whether the respondent was a supervising PHN or not)|
Significant difference between the groups (P < 0.05)