An evaluation of the factor loading standard errors within NOHARM using multidimensional item response theory data

NOHARM is frequently used within educational testing to determine a test's factor structure and estimate the multidimensional item response theory item parameters with dichotomous data. Simulation research revealed that this procedure performs superior to linear exploratory factor analysis methods with dichotomous data, although, little empirical work has evaluated its effectiveness to either conduct hypothesis tests or construct confidence intervals around the estimated factor loadings. The current study found that hypothesis testing and confidence intervals performed well for simple structure tests assuming the correlation between factors was reasonably small and the sample was large using the Promax rotation. Conversely, Varimax rotations only produced adequate results for simple structure tests when the correlation between factors was zero with larger sample sizes. For tests with approximate simple structure, unacceptable power or coverage rate results were obtained with both rotation procedures. Therefore, researchers using NOHARM to uncover a test's dimensional structure via inferential statistics and confidence intervals should use caution when interpreting the results, as most educational tests' possess large correlations between the factors and rarely exhibit pure simple structure.