Skip to main content

Table 1 Procedure of IDAS development and validation

From: Implementation Degree Assessment Sheet for Health Program in Japan by Customizing CFIR: Development and Validation

Phase

Step

Method

Purpose/adoption criteria

Phase 1: Item development

Step 1: Identification of domain and item

Choose an existing framework that has been validated.

Because CFIR (five domains, 38 constructs) is a meta-theoretical framework that integrates about 20 theories and models and has already been used and validated in many studies (over 2600 citations).

Step 2: Content validity

Forward translation and reconciliation: 4 individuals translated, consulted, and integrated.

Modify the contents to fit the context of Japanese health programs: 7 experts consulted.

To ensure the quality of the translation by adhering to the forward translation procedure by multiple people.

To ensure the quality of item selection and regeneration by Japanese experts with experience of health program implementation.

Phase 2: Scale development

Step 3: Pre-testing

Pre-test with the target group

To ensure the questions and answers are meaningful.

Adopt those with at least 70% agreement for each question of "easy to understand" and "think it is important”.

Step 4: Survey administration and sample size

National survey of the target population

To ensure the quality of reliability and validity by calculating a sufficient sample size for the survey and distributing it evenly throughout the country.

Step 5: Item analysis

Ceiling/floor effect

To examine whether there is a scale attenuation effect depending on the degree of asymmetrically of the frequency distribution.

Items with mean + 1SD < 5 or mean − 1SD > 0 are deleted.

Item-total correlation analysis

To check if any item in the set of tests is inconsistent with the averaged behavior of the others, and thus can be discarded.

Item-total correlation coefficient ≥ 0.5

Phase 3: Scale evaluation

Step 6: Tests of reliability

Cronbach’s coefficient alpha

To measure the internal consistency of IDAS.

Adopted at 0.8 or higher.

Split-half correlations (odd/even) method

To check the verification of stability of IDAS.

Adopted at the Spearman–Brown reliability coefficient 0.8 or higher.

Step 7: Tests of validity

Construct validity

To verify the model fitness of construct validity with a confirmatory factor analysis.

Check the goodness-of-fit index for fit in the same five domains as CFIR

GFI, CFI ≥ 0.9 RMSEA < 0.08

Criterion validity

To estimate the extent to which a test correlates with an established standard of comparison (RUC).

Correlation coefficient ≥ 0.5

Differentiation by “Known Groups”

To examine if the concept measured behaves as expected in relation to “known groups”. (Four groups depending on the years of experience and whether the respondent was a supervising PHN or not)

Significant difference between the groups (P < 0.05)